AD-A223  441 


AUGMENTING  U.S.  ARMY  WARRANT  OFFICER  EXPERTISE 
WITH  EXPERT  SYSTEMS 


A  research  project  submitted  to  the  faculty  of 
San  Francisco  State  University 
in  partial  fulfillment  of  the 
requirements  for  the 
degree 


Master  of  Science  in  Business  Administration 


DT1C 

ELECTE 

JUN291990 


D 


DISTRIBUTION 

STATEMENT  A  | 

Approval  for 
l  Dunuounoa 

public  release!  I 

Unlimited  J 

by 

GEORGIA  JOHANNA  SOLSETH 


San  Francisco,  California 
May  1990 


00  06  12  031 


STATEMENT  "A"  per  Maj.  J.  Whisker 
Total  Army  Personnel/TAPC-OPB-D 
200  Stovall  St.,  Alexandria,  VA  22332-0411 
TELECON  6/28/90  VG 


CERTIFICATION  OF  APPROVAL 


Accesion  For  j 

NTIS  CRA&I 
OTIC  TAB 
Unannounced 
Justification 

□ 

□ 

By  &AjC*JL 

<L 

Distribution  \ 

Availat  irly  Codes 


Dist 


M 


Avail  and/or 
Special 


I  certify  that  I  have  read  AUGMENTING  U.S.  ARMY  WARRANT 
OFFICER  EXPERTISE  WITH  EXPERT  SYSTEMS  by  Georgia  Johanna 
Solseth,  and  that  in  my  opinion  this  work  meets  the  criteria  for 
approving  a  research  project  submitted  in  partial  fulfillment  of 
requirements  for  the  Master  of  Science  in  Business  Administration 
degree  at  San  Francisco  State  University. 


Dr.  Gary  Hammerstrom 
Professor  of  BACS 


Dr.  Sam  Gill 
Professor  of  BACS 


ABSTRACT 


AUGMENTING  U.S.  ARMY  WARRANT  OFFICER  EXPERTISE 
WITH  EXPERT  SYSTEMS 

Georgia  Johanna  Solseth 
San  Francisco  State  University 
\  Spring  1990 

J 

The  purpose  of  this  project  was  to  determine  whether 
automated  expert  systems  can  be  effectively  used  to  augment  the 
diminishing  knowledge  base  of  personnel,  intelligence  and  supply 
warrant  officers  in  order  to  keep  valuable  and  needed  expertise 
within  the  U.S.  Army.  Based  upon  secondary  research,  a  model 
describing  the  selection  criteria  for  expert  system  applications  was 
developed.  This  model,  called  the  Expert  System  Application  Criteria 
Model  (ESAC)  ,  outlined  the  decision-making  thinking  processes,  the 
characteristics  of  the  decisions,  and  the  generic  functions  inherent  in 
successful  expert  system  applications.  A  survey,  with  questions 
designed  to  measure  the  variables  indicated,  was  sent  to  personnel, 
supply  and  intelligence  warrant  officers.  Using  summary  statistics,  a 
profile  for  each  warrant  officer  type  was  developed  and  compared  to 
the  ESAC  model.  It  was  concluded  that  expertise  used  by  intelligence 
warrant  officers  was  not  suitable  for  expert  system  application. 
Supply  and  personnel  warrant  expertise  are  prime  candidates  for 
expert  system  applications  but  will  require  additional  research 
before  expert  system  augmentation  is  accepted.  jp 

I  certify  that  the  Abstract  above  is  a  correct  representation  of  the 
content  of  this  research  paper. 
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CHAPTER  ONE: 

PROBLEM  DEFINITION  AND  STATEMENT 


Organizations  are  constantly  facing  a  tremendous  concern  -  the 
drain  or  loss  of  valuable  and  essential  resources  needed  to  accomplish 
their  goals.  Fortunately,  most  resources  can  be  readily  substituted 
and  replaced.  When  a  firm  does  not  have  enough  manpower  to 
complete  a  task,  more  individuals  are  hired.  When  a  company  has 
insufficient  funds  to  undertake  a  new  project,  a  loan  may  be  granted 
or  company  assets  sold.  Countries  are  looking  to  and,  in  many  cases, 
utilizing  alternate  energy  forms  to  replace  finite  fossil  fuel  sources. 
Although  not  always  simple  or  inexpensive,  there  are  usually 
systematic  and  proven  methods  and  approaches  to  replacing  limited 
resources. 

Because  of  its  unique  characteristics,  there  is  one  resource, 
however,  presenting  some  difficulty  in  effectively  replacing.  This 
resource  is  the  human  expert.  An  expert  can  be  defined  as  1 : 

"A  person  who,  because  of  training  and  experience,  is  able  to  do 
things  the  rest  of  us  cannot;  experts  are  not  only  proficient  but 

JP.E.  Johnson, "What  Kind  of  Expert  Should  a  System  Be?"  THE  JOURNAL  OF 
MEDICINE  AND  PHILOSOPHY,  Volume  8,  1983:  77. 
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also  smooth  and  efficient  in  the  actions  they  take.  Experts 
know  a  great  many  things  and  have  tricks  and  caveats  for 
applying  what  they  know  to  problems  and  tasks;  they  are  also 
good  at  plowing  through  irrelevant  information  in  order  to  get 
at  basic  issues,  and  they  are  good  at  recognizing  problems  they 
face  as  instances  of  the  types  with  which  they  are  familiar. 
Underlying  the  behavior  of  experts  is  the  body  of  operative 
knowledge  termed  expertise.” 

The  experience  and  know-how  developed  through  years  of 
working  in  a  particular  area  are  the  elements  which  hinder  the 
simple  replacement  of  experts.  In  addition,  the  quantities  of  true 
experts  in  a  given  field  are  generally  limited,  making  replacement  or 
substitution  difficult.  If  the  amount  or  quality  of  expertise  within  an 
organization  diminishes,  the  organization  could  potentially  lose  a 
valuable  and  essential  asset  needed  for  the  organization  to  function. 
Such  a  problem  is  currently  facing  the  United  States  Army. 

EBQBLEM  ..STATEMENT 

The  United  States  Army  is  a  vast,  bureaucratic  establishment 
with  many  facets.  This  service  must  coordinate  extensively  with 
many  internal  and  external  organizations  to  ensure  that  it  can 
economically  and  efficiently  feed,  bed,  clothe,  maintain,  arm,  and 
medically  take  care  of  itself.  In  order  to  do  this  effectively,  the  U.S. 
Army  has  warrant  officers. 

Warrant  officers  are  the  Army's  technical  experts.  They  are 
soldiers,  who  having  spent  numerous  years  employed  in  a  particular 
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field,  have  gained  tremendous  amounts  of  experience  and  knowledge 
valuable  and  necessary  to  the  United  States  Army.  As  a  result,  their 
expertise  is  rewarded  by  promotion  to  warrant  officer.  Their 
knowledge  and  skills  are  in  such  areas  as  medicine,  supply 
procurement  and  movement,  maintenance,  personnel,  military 
intelligence,  and  other  logistic  and  support  operations.  For  example, 
their  technical  expertise  ensures  that  wounded  and  sick  soldiers  are 
properly  diagnosed  and  treated,  supplies  are  received  and  distributed 
as  needed,  and  trucks,  weapons,  communications  gear  and  other 
equipment  remain  operational.  They  provide  the  necessary  expertise, 
recommendations,  and  knowledge  to  assist  new  soldiers  in  their 
development  as  future  experts  and  to  advise  senior  officers  in  making 
sound  strategic  and  operational  plans.  In  essence,  warrant  officers 
and  their  expertise  are  essential  to  maintaining  combat  readiness 
within  the  U.S.  Army  and  are  key  elements  in  ensuring  its 
effectiveness  and  success. 

Unfortunately,  the  Army's  knowledge  base  of  expertise,  the 
warrant  officer,  is  diminishing  and,  as  a  result,  so  is  the  combat 
effectiveness  of  the  U.S.  Army.  Colonel  Michael  Moseley,  Chief  of  the 
Warrant  Officers  Division,  states,  "The  numbers  of  warrant  officers  in 
the  United  States  Army  are  lower  now  than  they  have  been  in  many 
years  and  I  don't  expect  the  situation  to  improve.  Authorization 
levels  continue  to  be  reduced  in  all  areas."  1 


'Michcal  Moseley,  COLONEL,  U.S.  Army,  Telephone  Interview,  20  Feb.  1990. 
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BACKGROUND  INFORMATION 

The  reduction  in  the  number  of  authorizations  of  warrant 
officers  and,  therefore,  the  amount  of  available  expertise,  can  be 
attributed  to  three  major  events.  Currently  there  are  strong  political 
and  economic  pressures  within  the  United  States  to  reduce  the 
national  deficit.  These  pressures  have  lead  directly  to  the  need  for 
defense  spending  cuts.  The  military's  response  to  this  has  ranged 
from  projected  base  closures  to  reduction  of  overall  active-duty 
personnel  strength.  Unfortunately,  included  in  these  strength 
reductions  are  the  amount  of  warrant  officer  authorizations.  Fewer 
warrant  officers  result  in  less  amounts  of  available  technical 
expertise. 

Closely  related  is  the  changing  role  of  the  United  States  Army. 
The  Army  is  undergoing  major  structural  changes  in  response  to  the 
dramatic  moves  toward  democracy  taking  place  throughout  the 
world.  This  changing  role  has  lead  concerned  citizens  to  question  the 
need  for  an  army  as  it  now  exists.  As  a  result,  additional  political  and 
social  pressures  are  causing  a  push  toward  reductions  in  the  U.S. 
defense  posture  and  associated  spending.  Again,  cuts  would  lead  to 
personnel  reductions  and  the  problem  discussed  in  the  previous 
paragraph  is  amplified. 

The  final  event  effecting  the  quantity  of  expertise  has  been  the 
expanding  role  of  warrant  officers  from  technical  experts  to  leaders 
and  managers.  Four  years  ago,  warrant  officers  were  strictly 
technical  advisors  and  assistants,  whereas  commissioned  officers 
were  responsible  for  providing  the  role  of  leadership  and 
management.  Commissioned  officers  utilized  the  expertise  provided 
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by  warrants  to  lead  more  effectively.  In  1987,  warrant  officers  were 
given  the  option  of  becoming  commissioned  officers  and  thereby 
assuming  the  additional  role  of  a  leader  of  soldiers.  Approximately 
ninety  percent  of  all  warrant  officers  are  now  commissioned1. 
Although  this  statistic  is  advantageous  in  terms  of  the  warrant 
officers'  professional  development  and  the  army's  supply  of  leaders, 
it  indicates  that  significantly  less  time  is  available  for  experts  to 
spend  on  technical  matters.  The  additional  responsibility  of  leading 
and  managing  soldiers  has  made  the  already  overburdened  technical 
system  even  more  so. 

According  to  COL  Moseley,  this  diminishment  of  the  Army's 
knowledge  base  will  have  severe  negative  repercussions,  most 
immediately  in  the  personnel,  military  intelligence,  and  supply  areas 
where  severe  shortages  already  exist  2.  In  peacetime,  the  expertise  of 
warrant  officers  saves  money  and  time  by  ensuring  that  the  logistics 
and  support  systems  run  smoothly  and  effectively.  In  wartime,  the 
expertise  saves  lives.  If  this  expertise  was  not  available  or  even 
began  to  diminish,  as  it  appears  is  currently  happening,  the  United 
States  Army  would  lose  numerous  resources,  the  most  valuable  of 
these  being  human  life. 

RESEARCH  PAPER  PURPOSE 

Unfortunately,  there  appears  to  be  no  easy  solution  to  this 
problem.  The  warrant  officers'  years  of  experience  and  familiarity 


'Richard  Farrant,  Chief  Warrant  Officer  Four,  Personal  Interview,  13  Feb. 

1990. 

^Moseley 
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with  the  military  system  do  not  make  them  an  easy  asset  to  simply 
replace  with  another  individual.  Indeed,  since  authorizations  for 
warrant  officers  are  not  being  eliminated  entirely,  completely 
replacing  them  may  not  be  required.  What  appears  to  be  needed  is  a 
means  of  augmenting  their  expertise  that  would  increase  or,  at  a 
minimum,  stabilize  the  current  quality  levels  and  amounts  of 
expertise. 

The  purpose  of  this  paper  is  to  examine  an  information  system 
alternative  to  determine  what  solution,  if  any,  is  available  to  the 

Army  to  augment  the  type  of  human  expertise  present  in  warrant 
officers.  I  am  seeking  an  information  systems  solution  primarily  for 
two  reasons:  scope  and  area  of  interest.  Due  to  the  time  limit  imposed 
upon  this  research  project,  it  is  not  feasible,  or  even  possible,  to 
examine  all  the  viable  alternatives  for  augmenting  human  expertise. 

The  scope  of  possibilities  is  large,  ranging  from  management 

techniques  to  human  intelligence  development.  Therefore,  the  scope 
of  this  project  has  been  narrowed  to  include  only  information  system 
alternatives.  Additionally,  information  systems  is  a  subject  I  am 

interested  in,  as  well  as,  a  degree  I  am  working  towards.  It  is  only 
plausible  that  this  project  involve  the  research  of  an  information 
system  solution. 

I  have  imposed  one  additional  restriction.  Due  to  the  limited 
scope  and  time  restrictions  of  this  research  paper  and  the  vast  scope 
of  specific  areas  of  warrant  officer  expertise,  research  efforts  will  be 
directed  towards  determining  a  way  to  augment  human  expertise 
specifically  in  the  critically  short  areas  of  personnel,  supply,  and 
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military  intelligence.  A  discussion  of  current  approaches  to 
augmenting  human  expertise  follow  in  the  succeeding  chapter. 
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CHAPTER  TWO: 
TOPIC  DEVELOPMENT 


Within  the  past  several  years,  many  attempts  have  been  made 
to  emulate  or  capture  a  human’s  reasoning  and  thinking  abilities. 
"Artificial  Intelligence"  is  the  term  that  has  been  applied  to  this 
study.  Artificial  intelligence,  otherwise  known  as  AI,  is  a  research 
area  focused  on  producing  machines  exhibiting  behaviors  normally 
associated  with  human  intelligence1. 

One  of  the  most  immediately  promising  engineering  outgrowths 
of  artificial  intelligence  research  has  been  expert  systems.  An  expert 
system  is  an  intelligent  computer  program  that  uses  knowledge  and 
inference  procedures  to  solve  problems  that  are  difficult  enough  to 
require  human  expertise  for  their  solution2.  The  purpose  of  expert 
systems  is  to  augment  the  decision-making  processes  of  others  by 
capturing  and  making  available  the  decision-making  power  of  an 
expert.  An  expert  system  can  be  viewed  as  a  computer-based  tool 


Lawrence  K.  Laswell,  COLLISION:  THEORY  VS  REALITY  IN  EXPERT  SYSTEM 
(Reading:  QED  Information  Sciences,  1989),  8. 

2V.  Daniel  Chapman.  ARTIFICIAL  INTELLIGENCE  AND  EXPERT  SYSTEMS 
SOURCEBOOK,  (New  York:  Chapman  &  Hall,  1989),  26. 
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that  emulates  the  advise  that  a  human  expert  would  give  a  decision¬ 
maker;  the  software  equivalent  of  a  trusted  advisor,  a  colleague  or  a 
second  opinion  1 . 

Major  characteristics  of  expert  systems  include2; 

1)  the  ability  to  perform  at  the  level  of  an  expert, 

2)  representation  of  domain  specific  knowledge  in  the 
manner  in  which  the  expert  thinks, 

3)  incorporation  of  explanatory  mechanisms  and  ways  of 
handling  uncertainty  into  the  repertoire  of  the  system,  and 

4)  predilection  for  problems  that  can  be  symbolically 
represented. 

The  characteristics  of  this  kind  of  system  suggest  that  the 
following  key  points  are  necessary  for  understanding  and  building 
any  expert  system3; 

1.  There  must  be  at  least  one  expert  to  be  the  source  of  the 
knowledge  in  the  system. 

2.  The  expert  know-how  and  know-what  must  be  recorded  in 
systematic  structures  such  as  books,  computers,  and  videotapes. 

3.  The  expert  system's  rules  and  methodologies  must  be 
accessible  so  that  a  user  can  input  his  problem  data  as  well  as  have 
some  form  of  interactive  capability. 


1  James  A.  Mecklenberger,  "Expert  Systems:  The  Next  Technology 
Breakthrough  for  Education,"  FORTUNE,  Sep.  1987:  6. 

2 Jay  Liebowitz  and  David  A.  Desalvo,  STRUCTURING  EXPERT  SYSTEMS: 
DOMAIN,  DESIGN.  AND  DEVELOPMENT,  (New  Jersey:  Englewood-Cliffs,  1989),  4. 

3David  Bendel  Hertz,  THE  EXPERT  EXECUTIVE,  (New  York:  John  Wiley  & 
Sons,  1988),  57. 


4.  The  expert  system  must  be  open-ended;  that  is,  as  the  world 
changes,  it  must  be  possible  to  add  new  rules,  new  methods,  and  new 
ways  of  assimilating  and  dealing  with  data. 

Expert  systems  are  still  generally  thought  of  as  specialized 
systems,  using  special  computer  hardware  and  special  programming 
languages  such  as  LISP  and  PROLOG.  However,  simple  expert  systems 
are  appearing  on  standard-sized  personal  computers  using  expert 
system  "shells"  -  software  frameworks  for  developing  specific  expert 
system  applications1. 
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The  architecture  structure  of  expert  systems  is  somewhat 
reminiscent  of  human  cognitive  structures  and  processes2.  Just  as 
there  is  no  unique  set  of  traits  that  define  a  human  expert,  not  every 
expert  system  has  the  same  architecture.  However,  certain  generic 
attributes  appear  in  most  systems.  Though  a  variety  of  terminology 
is  used,  there  are  generally  three  components  of  expert  systems  as 
seen  in  Figure  1.  First,  there  is  the  KNOWLEDGE  BASE  which  consists 
of  the  set  of  facts  and  heuristics,  or  "rules  of  thumb",  about  the 
particular  domain.  It  is  comparable  to  the  long-term  memory  of  the 
expert.  The  second  element,  the  method  of  reasoning,  is  usually 
referred  to  as  the  INFERENCE  ENGINE.  The  inference  engine  is  the 
control  structure  that  allows  various  hypotheses  to  be  generated  and 

1  Edward  Yourdon,  MODERN  STRUCTURED  ANALYSIS,  (New  Jersey:  Yourdon 
Press,  1989),  33. 

2Larry  Bielawski  and  Robert  Lewand,  EXPERT  SYSTEMS  DEVELOPMENT: 
BUILDING  PC-BASED  APPLICATIONS,  (Reading:  QED  Information  Sciences,  1988), 
23. 


tested.  Finally,  some  type  of  dialogue  structure  or  USER  INTERFACE  is 
included  which  allows  the  user  to  interact  with  the  expert  system  and 
includes  such  aspects  as  a  method  of  information  acquisition,  natural 
language  interface  and  an  explanatory  interface1. 


FIGURE  1:  Expert  System  Architecture  and  Components 
(Adapted  from  Hunt,  1986) 


KNOWLEDGE  BASE 

Performance  of  the  expert  system  is  primarily  a  function  of  the 
size  and  quality  of  the  knowledge  base  that  it  possesses2.  A 
knowledge  base  is  the  information  portion  of  the  expert  system 


^iebowitz,  4. 
2Hunt,  26. 


which  normally  contains  both  facts  and  rules.  The  first  step  in  the 
creation  of  the  knowledge  base  is  to  acquire  the  knowledge. 

Usually  two  people  are  involved  in  the  knowledge  acquisition 
process:  the  expert  and  the  knowledge  engineer.  An  expert's  power 
or  expertise  is  generally  derived  from  extensive  domain  specific 
knowledge  such  as,  common  facts,  which  consist  of  widely  shared 
knowledge  that  is  accepted  by  the  professional  and  other  accepted 
sources  of  data,  and  heuristics,  or  knowledge  based  upon  good 
judgment  and  common  practice  or  rules  of  thumb1. 

Experts  rarely  have  the  self-awareness  to  recognize  the  diverse 
extent  and  interaction  of  their  knowledge2.  Knowledge  engineers,  who 
are  generally  programmers  or  engineering  analysts  with  the  ability  to 
transform  knowledge  from  an  expert  into  programming  code,  are 
needed  to  develop  the  knowledge  acquisition  process,  to  create 
reasoning  programs  to  utilize  the  knowledge  and  to  assure  the  logical 
collection  of  "expert"  knowledge. 

Knowledge  representation  depends  heavily  on  logic  to  organize 
the  knowledge  base.  It  is  a  difficult  task  because  of  the  complex 
intellectual  relationships  that  must  be  anticipated  and  accounted  for 
but  are  not  always  explicitly  stated.  The  knowledge  stored  should  be 
structured  in  a  format  which  facilitates  the  system's  reasoning.  The 
approaches  that  A1  developers  have  invented  for  representing 
knowledge  in  a  knowledge  system  include  array  structures,  semantic 
networks,  property  hierarchies,  various  list  structures  and  predicate 


•David  W.  Rolston,  ARTIFICIAL  INTELLIGENCE  &  EXPERT  SYSTEMS,  (New 
York:  McGraw-Hill,  1988)  9. 

2Hunt,  26 


calculus  set1.  Three  common  methods  of  representation  are  facts, 
rules,  and  frames. 

Facts  are  pieces  of  information  that  can  be  used  by  expert 
systems.  They  are  generally  statements  or  conditions  concerning  the 
world  and  as  such  they  are  mostly  transient  and  subject  to  change2. 
For  example: 

The  real  estate  market  is  good. 

The  sky  is  blue. 

Human  experts  can  often  make  decisions  based  on  incomplete, 
imprecise  or  uncertain  information.  Similarly,  facts  may  sometimes 
be  uncertain  or  inexact.  In  the  statement  above,  we  may  only  be  70 
percent  sure  that  the  real  estate  market  is  good  at  a  particular  point 
in  time.  One  way  of  expressing  uncertainty  is  to  state  it  in  terms  of 
one's  confidence  that  a  fact  or  rule  is  true.  To  represent  confidence  in 
a  fact,  a  confidence  or  certainty  factor  (CF)  is  used3.  In  order  for  an 
expert  system  to  get  reliable  information,  the  knowledge  base  must 
account  for  changes  over  time  and  uncertainty  factors.  Known  facts 
guide  the  reasoning  of  an  expert  system  since  they  determii.e  which 
rules  are  currently  applicable.  The  quality  of  the  reasoning  process  is 
greatly  affected  by  whether  all  the  relevant  facts  are  available  to  the 
system. 


1  Clyde  W.  Holsapple  and  Andrew  B.  Whinston,  BUSINESS  EXPERT 
SYSTEMS, (New  York:  Richard  D.  Irwin,  1897),  37. 

2Kamran  Parsaye  and  Mark  Chignell,  EXPERT  SYSTEMS  FOR  EXPERTS,  (New 
York:  John  Wiley  &  Sons,  1988),  41. 

3Bielawski,  36. 


Facts  by  themselves  cannot  be  used  for  reasoning.  Facts  relate 
together  with  rules  to  reason  and  derive  new  facts.  A  rule  is  a 

statement  which  normally  contains  a  condition  and  an  action.  Rules 
were  championed  as  a  means  of  representing  knowledge  in  a  form 

that  could  be  used  for  inference1.  Most  rules  can  be  expressed  in  the 
general  form: 

IF  premise 

THEN  conclusion; 

An  example  is: 

IF  sky-color  IS  blue  THEN  raining  is  FALSE. 

Facts  and  rules  are  important  knowledge  structures,  but  they  do 
not  provide  easily  accessible  knowledge.  A  frame  is  a  technique 
which  accomplishes  this  task.  It  does  this  by  assembling  objects, 

attributes,  and  values  into  special  structures  called  frames.  This 
representation  permits  the  hierarchical  information  about  object 
relationships  to  be  stored  in  the  knowledge  base2.  For  example 

consider  the  vehicle  frame  representation  in  figure  2. 


^arsaye  and  Chignell,  45. 

2Carl  Townsend  and  Dennis  Fuecht,  DESIGNING  AND  PROGRAMMING 
PERSONAL  EXPERT  SYSTEMS,  (New  York:  Tab  Books,  Inc.,  1986),  56. 
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FIGURE  2:  Vehicle  Frame  Representation 
(Adapted  from  Parsaye,  et.al.,  1988) 

A  frame  contains  related  bits  of  information  about  the  subject 
of  the  frame;  for  example,  vehicle  size,  options,  and  engine.  Lower 
level  frames  automatically  inherent  the  attributes  from  the  parent 
slots,  but  may  have  additional,  unique  attributes.  This  type  of 
representation  is  often  called  "object-oriented"  because  each  frame 
represents  an  object  type. 

Frame  representation  is  advantageous  if  the  hierarchical 
relationship  is  relatively  static  and  there  are  few  exceptions  to  the 
relationships. 


INFERENCE  ENGINE 

As  indicated  an  expert  system  typically  includes: 

FACTS,  which  are  elementary  pieces  of  knowledge, 
FRAMES,  used  to  organize  knowledge,  and 
RULES,  which  relate  facts  and  frames. 

Collectively,  these  facts,  rules  and  frames  are  referred  to  as  the 
knowledge  the  expert  system  possesses.  To  arrive  at  conclusions,  the 
expert  system  relates  pieces  of  knowledge  by  performing  inference  or 
deduction.  The  part  of  an  expert  system  that  performs  inference  is 
called  an  inference  engine1. 

The  concept  of  inference  is  based  on  simple  logic.  If  we  have  a 
rule  that  states:  IF  sky-color  IS  blue  THEN  raining  IS  false, 

and  we  have  the  fact:  sky-color  IS  blue, 

we  can  infer  the  fact:  raining  IS  false, 

by  applying  the  rule  to  the  fact.  The  inference  engine  then  places  the 
new  knowledge  in  the  knowledge  base  as  a  fact.  Any  subsequent  rule 
determinations  could  use  this  fact,  such  as:  IF  raining  IS  false  THEN 
air  pressure  IS  high. 

The  method  in  which  inferences  are  drawn  is  known  as  either 
FORWARD  CHAINING  or  BACKWARD  CHAINING.  Forward  chaining  is 
a  bottom-up  reasoning  process  that  begins  with  known  conditions  or 
facts  and  makes  inferences  to  reach  a  goal2.  It  focuses  on  the  premise 
rather  than  the  conclusions  of  the  rules.  The  inference  engine 
systemically  and  repeatedly  cycles  through  the  rules  until  one  is 
found  whose  premise  matches  a  fact.  This  rule  is  then  proved  or 


^arsaye  and  Chignell,  57. 
2Rolston,  10. 


"fired"  and  the  conclusion  is  added  to  the  database.  Forward  chaining 
is  sometimes  referred  to  as  a  "brute  force"  method  of  reasoning. 

In  backward  chaining  inference,  the  decision  process  is 
reversed.  A  top-down  reasoning  process,  the  system  accepts  a  goal  or 
hypothesis  and  tries  to  determine  which  other  goals  need  to  be 
proven  in  order  to  prove  the  initial  goal1.  If  these  goals  are  not 
immediately  available,  they  serve  as  new  hypotheses  that  require 
further  inference  and  so  on.  This  type  of  reasoning  is  referred  to  as 
"backward  chaining"  because  one  is  reasoning  backward  from  the 
hypotheses  to  the  data. 

A  classic  example  of  both  forward  and  backward  chaining  is  the 
medical  diagnosis  process.  The  doctor  examines  a  few  of  the  patient's 
symptoms  and  places  him/her  in  a  particular  diagnostic  category, 
such  as  orthopedic  injury  or  respiratory  ailment.  This  is  forward 
chaining  used  to  prioritize  goals.  Secondly,  the  doctor  chooses  from 
the  categories,  respiratory  ailment,  for  example,  and  examines  the 
listed  symptoms  to  see  if  they  correspond  to  the  patient's,  a  backward 
chaining  inference  process. 

The  most  versatile  system  can  use  both  techniques,  but  many 
expert  systems  utilize  only  one.  It  is  not  possible,  or  desirable,  to  say 
that  forward  chaining  is  better  than  backward  chaining  or  vice  versa. 
Indeed,  many  problems  can  be  solved  by  either  method  or  a 
combination  of  both,  depending  on  how  the  system's  rules  are 
structured2. 


'Parsaye  and  Chignell,  66. 

2Donald  T.  Hawkins,  "Artificial  Intelligent*  and  Expert  Systems  for 
Information  Professionals:  Basic  AI  Terminolog\ .  ON-LINE,  Sep.  1987:  91. 
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The  specific  nature  of  the  problem  may  determine  which 
problem-solving  method  is  best.  Backward  chaining  rules  are  useful 
for  diagnostic  and  deterministic  decision  processes;  i.e.  situations  with 
specified  sets  of  outcomes.  Forward  chaining  rules  are  most 
appropriate  for  monitoring  processes,  where  the  system  has  to 
intercept  a  set  of  incoming  facts.  The  "firing"  of  forward  chaining 
rules  often  triggers  a  series  of  backward  chaining  inferences1. 

Some  experts  further  classify  the  inference  engine  into 

consisting  of  two  components.  The  first  is  the  knowledge  base 

management  system  which  "manages"  the  knowledge  base  by 
automatically  organizing,  controlling,  propagating  and  updating  stored 
knowledge.  It  initiates  searches  for  knowledge  relevant  to  the  line  of 
reasoning  upon  which  the  inference  subsystem,  the  second 
component,  is  working.  The  symbolic  inference  subsystem  provides 
the  process,  backward  or  forwarding  chaining,  by  which  lines  of 

reasoning  are  formed2. 

USER  INTERFACE 

While  the  knowledge  base  and  the  inference  engine  are  the 

basis  for  the  experts  system's  reasoning,  the  user  interface  may  also 
influence  whether  the  expert  system's  advise  will  be  accepted  and 
put  to  use3. 


Laurence  E.  Huber  and  Rahn  Carlson  Huber,  "Cracking  the  Researcher's 
Monopoly  on  Expert  System  Expertise,"  INDUSTRIAL  ENGINEERING,  Jan.  1988: 
88. 

2Hunt,  28. 

3Parsaye  and  Chignell,  33. 
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If  the  system  is  difficult  for  the  intended  users  to  operate, 
chances  are  that  they  will  fall  back  on  their  old  ways  of  solving  the 
problem  and  not  use  the  system  at  all.  They  may  simply  not  be 
willing  to  expend  the  intellectual  effort  required  to  use  a  new 
technology.  Knowing  the  audience  is  an  important  first  step  in 
designing  an  interface.  Developers  cannot  create  an  appropriate 
interface  until  they  identify  the  range  of  experience  of  the  users1. 

Besides  the  ease  of  use,  another  aspect  of  an  effective  user 
interface  is  the  facility  of  explanation  of  the  system’s  reasoning. 
Although  studies  reveal  that  people  tend  to  have  a  lot  of  faith  in 
information  generated  by  a  computer,  no  serious  user  of  an  expert 
system  will  accept  its  recommendations  if  the  system  is  unable  to 
explain  how  it  reached  it  conclusions.  This  is  especially  true  if  the 
advise  of  the  system  differs  from  what  the  user  would  have  guessed. 
Because  a  solution  from  a  rule-based  system  is  achieved  by  a  series 
of  fired  rules,  the  system  can  provide  an  explanation  of  its  reasoning 
by  showing  what  the  rule  was  and  why  it  was  fired2. 

Another  request  that  a  user  may  make  of  the  system  is  for  the 
system  to  explain  why  it  is  asking  the  user  a  certain  question.  The 
user  may  feel,  for  example,  that  a  certain  question  is  irrelevant  to  the 
problem  at  hand  and  may  be  curious  as  to  why  the  expert  system 
even  raises  it.  A  well  designed  user  interface  can  accommodate  this 
type  of  request  for  additional  information. 


iBielawski,  194. 

2Robert  G.  Bowerman  and  David  E.  Glover,  PUTTING  SYSTEMS  INTO 
PRACTICE,  (New  York:  Van  Nostrand  Reinhold,  1988),  23. 
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EXPERT  SYSTEM  OPPORTUNITIES 

The  emerging  expert  systems  software  technology  can  provide  a 
range  of  benefits  in  various  applications  if  properly  employed. 

CAPTURE  KNOWLEDGE:  The  capture  of  a  specialist's 

knowledge  is  the  primary  justification  for  expert  system 
implementation.  This  benefit  is  the  result  of  condensing  the  specific, 
domain-dependent  problem-solving  knowledge  of  an  expert  or  group 
of  experts  into  a  computerized  system.  The  system  can  then  be 
queried  for  knowledge  facts  like  a  database  system.  It  can  be  asked 
to  make  inferences  based  on  the  knowledge  in  the  knowledge  base 
and  to  produce  justification  of  the  "reasoning"  strategies  and  decision 
paths  used  in  reaching  a  conclusion,  as  mentioned  in  an  earlier 
section. 

CODIFY  EXPERTISE:  Additional  rationale  for  expert  system 
implementation  is  its'  ability  to  codify  expertise.  This  codification 
helps  formalize  the  expert's  knowledge  making  it  available  to  a  wider 
audience.  Knowledge  codification  also  contributes  to  the  repeatability 
and  the  reliability  of  the  judgements  of  one  expert  or  to  produce 
consistency  among  a  group  of  experts1. 

TIME  SAVINGS:  Frequently  experts  spend  a  large  portion  of 

their  time  doing  only  moderately  specialized,  routine  work. 
Codification  can  produce  stand-alone  or  assistant  capacity  systems 
which  free  up  more  of  the  experts'  time2.  As  a  consequence  the 
experts  can  spend  the  majority  of  their  time  on  creative  endeavors  in 


Robert  A.  Edmunds,  THE  PRENTICE  HALL  GUIDE  TO  EXPERT  SYSTEMS,  (New 
Jersey:  Englewood  Cliffs,  1988),  21. 

2Edmunds,  22. 
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problem  solving  and  allow  routine  decisions  to  be  delegated  to  the 
machine. 

MAINTENANCE  AND  UPDATE  SAVINGS:  Another  goal  of 

expert  system  implementation  is  the  savings  of  maintenance  and 
updating  the  knowledge  base.  The  clear  distinction  of  facts,  heuristics 
and  deduction  knowledge  in  knowledge-based  systems  reduces 
maintenance  and  update  costs  because  the  effects  of  changes  are 
restricted  to  particular  knowledge  chunks  in  the  knowledge  base1. 

AVAILABILITY:  There  are  numerous  other  advantages  of 

expert  system  use,  but  two  stand  out  because  of  the  economics 
involved.  First,  many  simultaneous  users  can  be  accommodated  by 

an  expert  system.  Consultation  with  a  human  expert,  in  contrast,  is 
usually  one-to-one  or,  at  most,  in  small  groups.  Second,  because  the 
knowledge  base  is  contained  in  the  system,  the  expertise  remains 

available  even  when  the  human  expert  has  departed2. 

EXPERT  SYSTEM  LIMITATIONS 

Unfortunately,  expert  systems  have  several  restrictions.  These 
limitations  need  to  be  noted  before  a  determination  is  made  on 

whether  or  not  an  expert  system  is  the  advisable  solution  to  an 

individual  or  company's  decision-making  problem. 

CAPTURE  OF  LIMITED  TYPES  OF  KNOWLEDGE:  Expert 
systems  function  on  the  domain  of  extracted,  cognitive,  logical 
thinking  processes.  They  are  generally  not  adept  at  managing  highly 


Powerman,  13. 

2Dwight  B.  Davis,  "Artificial  Intelligence  Goes  to  Work,"  HIGH 
TECHNOLOGY,  Apr.  1987:  18. 
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sophisticated  sensory  input  or  mechanical  motor  output1.  Current 
expert  systems  cannot  tackle  broad,  multiple-direction  problem 

spaces.  In  addition,  only  knowledge  that  can  be  derived  from  the 
expert  and  implanted  into  the  system  can  be  used.  The  actual 

intelligence  cannot  be  extracted.  "Today's  expert  systems  fall  well 
short  on  dimensions  requiring  general  intelligent  behavior.  They  are 
more  akin  to  idiot  savants  than  to  real  human  experts,"  say 

knowledge-engineering  experts2. 

The  very  type  of  knowledge  used  by  human  experts  can  also  be 
a  limitation.  To  a  great  degree,  the  flexibility  of  human  problem 
solving  is  based  on  a  broad  background  of  ill-defined  common  sense 

knowledge.  Common  sense  is  found  to  be  important  when  performing 
cognitive  operations  in  either  familiar  or  unfamiliar  situations  and 
tends  to  play  a  greater  role  in  the  familiar  areas,  the  very  field  of  the 
expert's  specialty.  This  type  of  knowledge  is  difficult  to  codify  even  if 
its  influences  are  understood.  Common  sense  knowledge  and  broad 
ranging  conceptual  information  are  not  available  in  the  typical  expert 
systems  of  today. 

LIMITED  ERROR  AND  RECOVERY  CAPABILITIES:  Limited 
error  exception  and  recovery  is  another  difficulty  in  expert  systems. 
Expert  systems  typically  do  not  respond  well  to  situations  outside 
their  range  of  expertise.  Performance  can  degrade  and  result  in  a 
major,  project-halting  catch. 


Powerman,  17. 

2F.  Hayes-Roth  and  D.A.  Waterman,  BUILDING  EXPERT  SYSTEMS,  (Reading: 
Addison-Wesley,  1983),  55. 
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TIME-CONSUMING  TASK:  Additional  disadvantages  have 

been  discovered  by  companies  utilizing  expert  system  technology. 
Expert  systems  technology  can  be  expensive,  especially  when 
extracting  knowledge  from  experts  is  time  consuming.  Some  thought 
processes  have  become  second  nature  and,  as  a  result,  require  a 
tremendous  amount  of  time  and  energy  to  extract  and  codify. 
Knowledge  development  can  sometimes  take  up  to  a  year  or  more1. 

RELIANCE  ON  EXPERTS:  Reliance  on  expert  opinion  has 
inherent  limitations  as  well.  One  is  "when  you  emulate  experts,  you 
also  emulate  their  faults."  Lacking  common  sense,  an  expert  will  not 
do  well  in  unanticipated  circumstances2.  Similarly,  some  experts  lie 
because  they  feel  threatened,  see  change  coming,  and  want  to  protect 
themselves  and  their  opinions.  There  is  the  additional  difficulty  of 
conflicting  advise  among  experts,  the  tendency  of  experts  to  display 
tunnel  vision  and  the  experts'  limits  in  depth  and  breath  of  skills3. 

As  noted,  experts  in  knowledge-based  systems  believe  expert 
systems  are  capable  of  performing  within  a  specific,  logical-oriented 
realm  of  expertise.  The  deeper  values  of  human  consciousness, 
interwoven  as  they  are  into  the  phenomenon  and  expressions  of 
human  intelligence,  however,  are  beyond  the  range  of  anything 
currently  available  in  AI4. 


1  Huber  and  Huber,  59. 

2Andrew  Kupfer,  "Now,  Live  Experts  on  Floppy  Disks,"  FORTUNE,  Oct.  1987: 
78. 

3Randall  Davis,  "Expert  Systems:  How  Far  Can  They  Go?"  AI  MAGAZINE, 
Summer  1989:  66. 

4Bowerman,  18. 
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PROBLEM  AND  DOMAIN  IDENTIFICATION 

Given  the  limitations  of  current  technology  in  expert  systems, 
the  first  critical  aspect  of  application  development  is  choosing  a 
problem  of  appropriate  degree  of  difficulty.  One  expert  states  the 
three  important  rules  in  developing  expert  systems  is  "pick  the  right 
problem,  pick  the  right  problem,  and  pick  the  right  problem1." 

Before  development  of  an  expert  system  begins  two 
determinations  must  be  made.  A  problem  must  be  selected  and  its 
domain  clearly  established.  The  need  for  a  definite  establishment  of 
a  domain  is  due  to  the  nature  of  expert  systems.  In  general,  the 
knowledge  of  a  expert  is  useful  only  for  a  particular  type  of  problem. 
Once  the  nature  of  the  problem  to  be  solved  crosses  the  boundaries  of 
the  domain,  the  expert  system  becomes  ineffective2.  As  one  expert 
states,  "the  task  must  have  a  domain  to  avoid  the  combinatorial 
explosion  of  the  alternatives3." 

A  variety  of  vague  economic  and  situation  dependent  criteria 
have  been  written  about  assessing  the  viability  of  programming  an 
expert  system  for  a  particular  domain  or  problem.  However,  the 
following,  well-established  rules  are  generally  provided  for  guidance4: 

•  The  problem  should  have  a  well-bounded  domain,  neither  too 
complex  or  too  simple.  Selecting  too  large  a  problem  or  one  with  few 
test  cases  could  lead  to  disastrous  results.  Picking  too  trivial  a 


^iebowitz,  3. 

2Huber  and  Huber,  59. 

3Liebowitz,  4. 

4Edith  Myers,  Expert  Systems:  Not  for  Everyone."  DATAMATION,  May  1986: 
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problem  will  leave  managers  and  users  unimpressed  and  will 
probably  not  be  worth  the  effort  for  development. 

•  The  domain  must  be  decision-oriented  as  opposed  to  creative 
in  nature.  The  decision  maker  must  have  some  specialized  expertise 
to  be  effective  in  the  task. 

•  The  problem  requires  the  development  of  multiple 
hypothesis  or  alternatives  and  the  need  to  explore  them  rapidly.  The 
decision  should  be  goal-oriented,  with  the  expert  decision  maker 
deciding  among  concrete  outcomes;  e.g.  Fault  A,  Fault  B,  or  Fault  C. 

•  The  problem  requires  the  use  of  experience  rather  than 
common  sense  as  the  key  to  the  solution. 

•  The  number  of  outcomes  should  be  discrete  and  fairly  small. 
Continuous  goals,  such  as  "invest  20%  in  stock,  11%  in  real  estate,  ect.," 
are  difficult  to  model.  Similarly,  the  amount  of  programming 
increases  with  the  number  of  goals  in  the  domain. 

•  The  decisions  being  modelled  should  be  made  fairly  often 
and  require  a  few  minutes  to  a  few  hours  for  a  human  to  make.  A 
prime  example  of  short-duration  decision  making  is  troubleshooting. 

•  The  decision  is  better  made  using  expert  knowledge  rather 
than  non-expert  knowledge. 

Guidelines  for  expert  systems  use  include  solving  problems 
where  people  have  experience  that  cannot  be  modeled  quantitatively. 
Expert  systems  are  not  for  problems  that  are  unsolvable  or  problems 
that  can  be  solved  by  mathematical  models.  Some  opportunities  for 
use  of  expert  systems  lie  in  areas  where  there  are  an  insufficient 
number  of  experts  to  provide  needed  knowledge,  where  there  is 
inconsistent  job  performance  and  poor  quality  of  work,  and  where 
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there  is  a  predominance  of  knowledge  intensive  as  opposed  to  clerical 
tasks1.  In  these  areas,  expert  systems  can  incorporate  an  expert's 
knowledge,  provide  multiple  access  to  the  knowledge,  and  ensure 
consistent  high  quality  expertise  is  available  as  needed. 


CURRENT  EXPERT  SYSTEM  APPLICATIONS 

Based  upon  the  opportunities  and  limitations  presented,  expert 
systems  should  be  viewed  as  offering  new  ways  of  doing  old  things: 
to  solve  problems  that  thwart  data  processors,  to  preserve  perishable 
expertise,  to  distribute  expertise,  to  fuse  multiple  sources  of 
knowledge,  to  convert  knowledge  into  a  competitive  edge  and  to  alter 
business  common  sense  and  perspective. 

Today,  users  of  expert  system  applications  are  doing  just  these 
things.  In  figure  3,  the  generic  categories  of  expert  systems 
applications  are  presented.  Other  experts  include  additional 
categories  such  as  fault  isolation,  scheduling,  analysis,  maintenance, 
configuration,  and  targeting  resource  allocation2.  These  types  of 
problem  categories  may  be  found  in  such  areas  as  agriculture, 
chemistry,  computer  systems,  electronics,  engineering,  geology, 
informatioi  management,  law,  manufacturing,  mathematics,  medicine, 
meteorology,  military  science,  physics,  process  control,  or  space 
technology  areas3.  In  1987,  companies  spent  an  estimated  $250 
million  on  software  for  developing  expert  systems.  This  number  is 

1  Myers,  29. 

2Liebowitz,  6. 

3D.A.  Waterman,  A  Guide  to  Expert  Systems,  (Reading:  Addison-Wesley, 

1986),  40. 
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projected  to  increase  to  $700  million  for  expert  systems  development 
in  19901. 
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Inferring  likely  consequences  of  given  situations 

Diagnosis 

Inferring  system  malfunctions  from  observables 

Design 

Configuring  objects  under  constraints 

Planning 

Designing  plans  of  action 

Monitoring 

Comparing  observations  to  anticipated  outcomes 

Debugging 

Prescribing  remedies  for  malfunctions 

Repair 

Executing  a  plan  to  administer  a  prescribed  remedy 

Instruction 

Diagnosing  and  rectifying  student  performance 

Control 

Controlling  overall  system  behaviors 

FIGURE  3:  Generic  Categories  of  Expert  Systems 
(Adapted  from  Hayes-Roth,  et.  al.,  1983) 

Examples  of  some  of  the  current,  successful  applications  in 
these  areas  follow. 

•  The  greatest  use  of  expert  systems  has  been  in  China  where 
knowledge  engineers  are  capturing  the  knowledge  of  rural,  herbal 
doctors,  a  dying  profession  that  goes  back  millennia,  and  distributing 
this  expertise  throughout  the  country  with  expert  systems2. 


1  Kupfer,  69. 

2Kupfer,  70. 
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•  Digital  Equipment  Corporation  (DEC)  uses  an  expert  system 

called  XCON  (eXpert  CONfigurer)  to  help  configure  its  complex 
minicomputer  systems.  Each  system  must  be  specifically  designed  to 
meet  an  individual  customer's  requirements.  When  this  task  was 
performed  by  highly-trained  engineers,  few  of  the  systems  produced 
were  completely  correct.  Often  something  a  simple  as  an  improper 
cable  costing  a  few  dollars  would  delay  the  implementation  of  a 
$100,000  system.  Today,  most  systems  go  out  correct.  By  some 

estimates  DEC  is  saving  about  $25  million  a  year  using  expert  system 
technology1. 

•  Several  large  insurance  companies  are  using  expert  systems 

software  to  help  handle  their  business.  A  system,  called  The 

Underwriter  Advisor,  is  being  used  to  act  as  a  consultant  to  the 

underwriter.  As  would  be  the  case  with  a  human  consultant,  the 

underwriter  is  still  free  to  accept  or  reject  the  advise  of  the  expert 
system2. 

•  The  company.  Combustion  Engineering,  utilizes  expert  systems 
to  rapidly  redesign  the  parts  it  uses  to  manufacture  coal-burning 
boilers  for  power  utilities.  The  company  estimates  that  the  system 
compresses  1000  hours  of  engineering  time  into  about  100  hours; 
productivity  gains  of  ten  to  one3. 

•  At  American  Express,  expert  systems  decide  if  someone 
making  a  purchase  is  a  fraud  or  a  deadbeat.  It  allows  less 


Edmunds,  4. 

2Linda  Runyan,  "Hoi  Technologies  for  1989,"  DATAMATION,  Jan.  1989:  18. 
3Runyan,  18. 
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experienced  workers  to  arrive  at  credit  judgements  that  are  as  sound 
as  an  experts1. 

•  The  United  States  Department  of  Defense  (DOD)  has  defined  AI 

as  one  of  the  one  most  critical  technologies  to  pursue  in  the 
remainder  of  this  century.  Through  the  Defense  Advance  Projects 
Research  Agency,  the  DOD  is  channelling  hundreds  of  millions  of 
dollars  into  universities  and  private  industry  to  create  next 

generation  machines  and  software2.  Some  examples  of  where  the 
military  is  utilizing  expert  systems  are  in  the  Navy,  where  they  select 
the  proper  types  of  acoustic  buoys  to  throw  into  the  water  to  locate 
enemy  submarines,  or  in  the  Air  Force,  where  fighter  jets  are 
supplied  with  systems  to  identify  aircraft3. 

•  Certain  hospitals  within  the  United  States  are  using  expert 

systems  to  diagnose  and  prescribe  cures  for  specific  diseases.  The 
best  known  system  is  MYCIN,  which  provides  advise  to  physicians  in 
treating  patients  with  bacterial  infections4.  Another  expert  system 

program  called  PUFF,  is  diagnosing  certain  lung  diseases  with  almost 
90  percent  accuracy.  The  system  compares  various  measures  of  lung 
functions  against  profiles  drawn  from  case  histories  of  victims  with 

lung  disease5. 

Expert  systems  have  shown  vast  commercial  promise  in  the 
past  few  years.  In  fact,  expert  systems  may  eclipse  all  other  areas  of 
AI  research  and  development  activities.  As  illustrated,  many 


'Kupfer,  69. 
2Edmunds,  6. 
3Kupfcr,  70. 
4Hawkins,  93. 
5Edmunds,  12. 


successful  expert  systems  have  already  been  placed  into  production. 
However,  there  is  general  agreement  that  these  early  systems  have 
barely  scratched  the  surface  of  what  is  almost  surely  destined  to 
become  a  common  place  technology  by  the  end  of  the  century. 

RESEARCH  QUESTION 

The  question  appears  to  be  whether  or  not  expert  systems  can 
truly  augment  or  replace  human  expertise  as  we  know  it.  Some 
examples  have  been  provided  which  illustrate  where  expert  systems 
are  currently  working  with  apparent  success.  In  these  applications, 
expert  systems  have  augmented  the  work  of  an  expert.  Among  these 
applications  there  appear  to  be  several  common  factors.  Essentially, 
existing  expert  system  applications  have  replaced  or  augmented 
human  experts  who  used  similar  thinking  processes  in  their  decision 
making,  made  similar  types  of  decisions,  and  performed  similar  types 
of  functions. 

One  way  to  determine  if  a  particular  type  of  expertise  can  be 
supported  by  an  expert  system  would  be  to  determine  if  the 
characteristics  of  their  decision  making  are  equitable  with  the 
decision  making  elements  of  existing  expert  systems. 

Although,  it  appears  as  if  little  or  no  research  has  been 
conducted  addressing  this  paper's  specific  concern  -  the  augmentation 
of  U.S.  Army  warrant  officer  expertise  -  it  has  been  demonstrated 
that  human  expertise  can  be  supplemented  in  certain  areas  and 
under  certain  conditions.  Whether  or  not  these  conditions  exist  within 
the  U.S.  Army  warrant  officer  system  still  remains  undetermined. 
Warrant  officers  are  the  Army’s  experts  and  fit  the  term  "expert"  is 
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every  sense  of  the  word.  They  have  extensive  training  and 
experience  in  a  particular  field  that  allows  them  to  make  quick  and 
proficient  decisions  or  recommendations.  Therefore,  it  would  follow, 
that  the  expertise  of  a  warrant  officer  is  not  unlike  that  of  others 
experts,  to  include  those  that  are  supported  by  expert  system 
applications. 

This  paper  will  attempt  to  determine  if  this  is  indeed  the  case, 
by  answering  the  following  research  question: 

Can  automated  expert  systems  be  effectively  and  efficiently 
used  to  augment  the  diminishing  knowledge  base  of  personnel, 
intelligence,  and  supply  warrant  officers  in  order  to  keep  valuable 
and  needed  expertise  within  the  U.S.  Army’s  organization? 

This  research  question  will  be  answered  by  determining 
whether  or  not: 

1)  The  type  of  cognitive  decision  processes  used  by  these 
particular  warrant  officers  in  their  daily  decision-making  is  suitable 
for  use  in  expert  systems, 

2)  The  type  and  characteristics  of  decisions  made  by 
warrant  officers  are  equitable  to  the  characteristics  of  decisions  that 
can  be  proposed  by  expert  systems,  and 

3)  The  daily  functions  performed  by  warrant  officers  fall 
within  the  current  categories  of  proven  working  and  effective 
applications  of  expert  systems. 
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CHAPTER  THREE: 
ANALYSIS 


Based  upon  the  available  secondary  research  in  successful 
expert  system  applications  outlined  in  Chapter  Two,  it  is  possible  to 
create  a  model  which  profiles  appropriate  expert  system  application 
problem  selection  criteria.  The  research  indicated  that  the 
appropriate  expert  system  problem  is  primarily  a  function  of  the  type 
of  thinking  processes  used  in  decision-making,  the  characteristics  of 
the  decision,  and  the  functions  where  the  decision-making  occurs. 
This  functional  relationship  can  be  seen  in  a  mathematical  view  of  the 
model  in  Figure  4. 


C  —  f (  A |  , A 2  ,A3) 

Where, 

C  =  Expert  System  Selection  Criteria 
A1  =  Type  of  Thinking  Processes  Used 
A 2  =  Characteristics  of  Decisions 
A3  =  Functions  Performed  in  Decision 
Making 


Figure  4:  Mathematical  Model:  Expert  System  Application  Criteria 
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In  the  decision-making  thinking  process,  known  facts  and  the 
use  of  experience  or  information  available  through  printed  material, 
are  best  suited  for  expert  system  applications.  The  use  of  common 
sense  in  decision  making  is  not  easy  to  codify.  Additionally, 
mathematical  models  are  not  suitable  for  expert  systems.  The  types 
of  decisions  that  are  easily  generated  by  expert  systems  are  generally 
routine,  non-creative,  and  require,  on  the  average,  1  minute  to  3 
hours  to  make.  Expert  systems  work  well  in  situations  that  are 
decision-oriented,  utilize  an  expert’s  knowledge  in  the  decision 
solution  and  perform  a  variety  of  functions  listed  in  Figure  3,  Chapter 
2.  These  specific  values,  determined  by  this  researcher’s  analysis  of 
available  expert  system  information  and  research,  can  been  seen  in 
Appendix  A. 

In  order  to  answer  the  research  question:  whether  or  not 
automated  expert  systems  can  be  effectively  and  efficiently  used  to 
augment  the  diminishing  knowledge  base  of  personnel,  intelligence, 
and  supply  warrant  officers  in  order  to  keep  valuable  and  needed 
expertise  within  the  U.S.  Army's  organization,  a  survey  was  created 
and  distributed  to  supply,  intelligence,  and  supply  warrant  officers  in 
order  to  provide  data  on  the  variables  indicated  earlier.  The 
variables  were  measured  and  then  applied  to  the  Expert  System 
Application  Criteria  Model  (Appendix  A).  The  methodology  of  the 
process  follows. 
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RESEARCH  DESIGN 

In  this  paper,  the  independent  variable  is  the  warrant  officer's 
branch  speciality:  personnel,  intelligence,  or  supply.  The  dependent 
variables  are  the  thinking  processes  utilized,  the  decision 
characteristics,  and  the  functions  performed.  Since  the  research 
question  can  be  answered  by  simply  measuring  these  dependent 
variables,  and  not  by  manipulating  them,  an  Ex  Post  Facto  research 
design  was  selected. 

Specifically,  a  survey  design  was  chosen  for  three  reasons. 
First,  the  survey  design  allowed  this  researcher  to  study  and  describe 
three  somewhat  large  populations  fairly  quickly.  With  the  time 
constraints  of  this  project  it  was  essential  to  have  a  quick  method  of 
data  collection.  Second,  survey  costs  are  relatively  low  compared  to 
other  designs,  solving  another  problem  constraint  of  the  project. 
Finally,  surveys  have  been  used  successfully  to  describe  population 
characteristics  which  is  the  primary  purpose  of  this  paper1. 

MEASUREMENT  TOOLS 

The  design  of  this  project's  questionnaire  (see  Appendix  B)  was 
accomplished  with  the  following  factors  in  mind: 

1)  Effective  measurement, 

2)  Data  validity,  and 

3)  Physical  format. 


1  Duane  Davis  and  Robert  Cosenza,  BUSINESS  RESEARCH  FOR  DECISION 
MAKING,  (Boston:  PWS-Kcnt  Publishing,  1988),  111. 
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In  order  to  measure  the  dependent  variables,  elements  in  the 
Expert  System  Application  Criteria  Model  were  incorporated  into  the 
questionnaire.  The  elements  of  the  model  and  corresponding 
questionnaire  measurements  are  depicted  in  Appendix  C.  In  most 
cases,  ordinal  measurements,  in  the  fo  nt  of  Likert  Scales,  which 
provide  information  about  the  relative  amount  of  some  trait 
possessed  by  an  object  or  indiyidual,  were  used.  Some  nominal 
measurements  were  also  designed  into  the  questionnaire  for 
classification  usage. 

In  order  to  ensure  data  validity,  multiple  procedures  were 
undertaken.  An  extensive  search  of  the  literature  was  conducted  in 
the  development  of  the  descriptive  model.  As  a  result,  during  the 
survey  design  all  elements  of  the  model  were  included  in  order  for 
each  to  be  measured  and  to  ensure  content  validity.  Construct 
validity  was  added  into  the  survey  by  including  two  different 
measurements  which  measure  essentially  the  same  concept  or  a 
measurement  scale  that  could  be  differentiated  from  another  scale 
measuring  a  different  concept.  This  method  of  data  validity  was 
selected  primarily  because  time  and  cost  limitations  did  not  allow  for 
survey  pre-testing.  A  listing  of  validity  checks  is  provided  in 
Appendix  D. 

The  design  of  the  physical  format  was  developed  essentially 
with  the  thought  of  receiving  as  many  valid  responses  as  quickly  as 
possible.  The  size,  therefore,  was  limited  to  one  page,  front  and  back. 
The  questions  were  presented  in  increasing  order  of  difficulty,  so  as 
to  not  frighten  a  respondent  away.  Likewise,  personal  data  was  the 
last  section  of  information  to  complete.  With  the  exception  of  the 
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personal  data,  all  questions  were  closed-ended  for  ease  in  respondent 
answering  and  researcher  coding  and  analysis. 

A  review  of  Appendix  C  indicates  that  a  few  questions  were 
included  in  the  survey  that  do  not  serve  any  useful  measurement 
purpose.  These  were  included  into  the  survey  in  error  as  the 
researcher  had  originally  thought  to  also  measure  conditions  under 
which  expert  system  use  is  beneficial,  i.e.  when  an  expert  departs  an 
area  without  an  immediate  replacement.  After  working  with  the 
Expert  System  Application  Criteria  Model  it  was  determined  that 
conditions  where  expert  systems  were  helpful  was  irrelevant  to 
problem  selection.  Retroactively,  however,  these  irrelevant  questions 
did  serve  a  purpose.  This  survey  was  distributed  to  the  warrant 
officers  without  an  explanation  of  the  research  question  it  was 
attempting  to  answer.  The  subject  was  kept  unknown  due  to  the  fact 
that  the  researcher  thought  the  responses  might  to  skewed  in  one 
direction  or  another  because  of  the  potential  effect  an  expert  system 
may,  or  may  not,  have  on  a  warrant  officer’s  perceived  employment. 
The  questions  that  served  no  measurement  purpose  helped  broaden 
the  types  of  questions  asked  and,  therefore,  hide  the  underlying 
meaning  of  the  relevant  questions. 

SAMPLING  PROCESS 

An  analysis  of  three  populations  was  conducted  for  this 
research:  personnel  warrant  officers,  intelligence  warrant  officers, 
and  supply  warrant  officers.  The  three  populations  were  finite  with 
448,  144,  and  302  units  per  population,  respectively. 
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The  sampling  design  used  was  systematic  sampling.  The 
Warrant  Officers  Branch  of  the  U.S.  Army  Personnel  Command 
provided  complete  listings  of  the  three  populations.  Using  natural 
ordering,  a  randomly  selected  start  point,  and  the  sampling  ratio  for 
the  particular  population,  a  sample  selection  of  warrant  officers  was 
made. 

Several  factors  were  considered  in  determining  the  sample  size, 
the  most  important  of  these  was  having  confidence  that  the  sample 
captures  the  true  and  precise  population  parameters.  Additionally, 
past  response  rates  in  this  type  of  project  were  considered  in  order  to 
ensure  a  significant  number  of  responses  were  available  for  analysis. 
A  sample  size  of  30  percent  of  the  population  was  selected  with  the 
exception  of  the  intelligence  population,  which  had  a  relatively  small 
population  to  select  from.  A  larger  percentage  of  the  population  was 
selected  in  order  to  ensure  a  significant  amount  of  responses  were 
received  for  analysis.  A  summary  of  the  sampling  statistics  is 
provided  in  Appendix  E. 

DATA  ANALYSIS 

As  indicated  at  the  beginning  of  this  chapter,  a  descriptive,  and 
somewhat  theoretical  model  exists  for  determining  if  an  application  is 
suitable  for  expert  system  implementation.  The  basic  premise  of  this 
paper's  analysis  methodology  was  to  develop  a  profile  of  thinking 
processes,  decision  characteristics,  and  functions  performed  for  each 
population,  and  then  to  compare  these  profiles  to  the  Expert  System 
Application  Criteria  Model.  If  the  variables  in  the  profile  compare 
favorably  with  the  model  variables  a  determination  can  be  made  that 
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the  decisions  and  decision-making  processes  utilized  by  that 
population  are  suitable  for  expert  system  application. 

Data  Statistics:  Due  to  the  descriptive  nature  of  the  research 

and  the  goal  of  describing  the  criteria  of  suitable  expert  system  use, 
summary  statistics  were  determined  to  be  the  most  appropriate 
analytical  method  for  this  project. 

Data  provided  by  nominal  measurement  was  used  by 
determining  the  most  frequent  appearing  response,  or  the  Mode. 
Data  provided  through  the  use  of  ordinal  measurement  measured 
Central  Tendency  and  Dispersion,  by  computing  means,  medians, 
modes,  ranges,  and  standard  deviations  of  each  variable  measured. 
The  specific  types  of  summary  and  tabulation  statistics  can  be  seen  in 
the  appendices  where  the  primary  data  for  each  population  is 
presented  (  Appendices  F,  G,  or  H). 

Profile  Development:  The  summarized  statistics  were  then 

used  to  develop  a  profile  of  each  populations’  decision-making 
variables.  Elements  added  to  each  profile  were  determined  by: 

1)  Mode  responses  for  Part  I. 

2)  Responses  of  greater  than  3  for  Part  II  questions  were 
defined  as  a  positive  response. 

3)  Responses  of  less  than  3  for  Part  11  questions  were 
defined  as  a  negative  response. 

4)  Responses  of  3  for  Part  II  questions  indicated 
irrelevancy  and,  therefore,  were  not  added  to  the  profile. 


5)  Responses  of  2.5  or  greater  for  Part  III  questions  were 
defined  as  a  positive  response. 

6)  Responses  of  less  than  2.5  for  Part  III  questions 
were  defined  as  functions  that  were  not  significantly  utilized  and, 
therefore,  were  not  added  to  the  profile. 

The  sample  profiles  of  personnel,  intelligence,  and  supply 
warrant  officers  populations  are  provided  in  Appendices  I,  J,  and  K, 
respectively. 

Model  Comparison:  As  a  final  step  in  the  data  analysis,  each 
profile  element  and  corresponding  measurement  was  compared  to 
the  Expert  System  Application  Criteria  Model.  The  results  of  this 
comparison  as  well  as  all  the  analysis  mentioned  earlier  is  presented 
in  the  following  section. 

RE$VLT§ 

The  Personnel  Warrant  Officer  Profile:  The  profile  of  the 
personnel  warrant  officer  was  developed  using  a  sample  size  of  67. 
The  total  number  of  responses  was  79,  however,  12  responses  were 
thrown  out  because  of  the  incompleteness  of  questionnaires  and 
possible  validity  problems.  As  mentioned  earlier  when  discussing 
validity  checks,  a  response  of  one  type  of  question  indicated  that  a 
particular  response  should  follow  for  another  type  of  question  (see 
Appendix  D).  Three  responses  were  eliminated  because  of 
incompleteness  while  the  rest  were  disregarded  due  to  possible 
validity  errors.  Analysis  of  the  remaining  responses  indicated  that 
the  thinking  process,  decision  characteristics,  and  functions 
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performed  and  utilized  by  personnel  warrant  officers  paralleled  the 
model  in  all  but  three  areas. 

In  determining  the  thinking  process  used,  76  percent  of  the 
respondents  indicated  that  past  experience  was  the  primary  factor 
used  in  making  decisions.  The  second  most  frequent  response,  of 
which  19  percent  of  the  respondents  replied,  was  researching  army 
regulations  and  manuals.  Both  of  these  methods  are  deemed 
appropriate  when  determining  applicable  expert  system  applications. 
A  mere  4  percent  of  the  respondents  indicated  common  sense  as  the 
primary  element  used  in  decision  making.  The  extent  of  experience 
used  ranged  from  "almost  never"  to  "almost  always."  The  mean, 
median,  and  mode  were  3.0  in  all  cases,  proving  essentially  nothing 
one  way  or  another.  Logically,  this  response  should  have  averaged 
above  a  3.0  as  determined  by  the  large  percentage  of  "experience" 
responses  in  the  earlier  question.  This  conflict  could  be  explained  by 
the  somewhat  confusing  wording  of  the  question.  This  problem  will 
be  discussed  later  in  Chapter  4  under  Possible  Problem  Areas. 
Ninety-four  percent  of  the  respondents  indicated  that  common  sense 
was  used  in  their  decision  making  from  a  range  of  "never"  to 
"sometimes."  This  data  supports  the  earlier  response  of  only  4 
percent  who  indicated  that  common  sense  was  the  primary  factor  in 
their  decision  making  process.  The  mean  response  for  use  of 
mathematical  models  was  2.58  indicating  that  respondents  used 
models  less  often  than  "sometimes."  The  final  variable  of  the  thinking 
process,  the  use  of  known  facts,  measured  21  percent  of  the 
responses.  Seventy-five  percent  indicated  that  both  facts  and 
intuition  were  used  in  the  process.  Thus  it  can  be  concluded  that  in 
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the  decision  making  thinking  process,  experience  is  used,  common 
sense  and  mathematical  models  are  not  significantly  used,  and  known 
facts  are  taken  into  consideration  but  only  in  conjunction  with 
intuition  or  personal  insight. 

In  determining  the  characteristics  of  decisions  made  by 
personnel  warrants,  78  percent  of  the  respondents  indicated  that 
most  decisions  take  one  minute  to  three  hours  to  make  with  the 
majority  of  those  responses  (76  percent)  being  1  to  59  minutes. 
Likewise,  the  mean  responses  to  the  regularity  of  the  decision  was  a 
4,  or  an  "almost  always"  response.  The  mean  responses  to  the 
creativity  of  decisions  was  2.97.  Here  responses  ranged  from 
"always"  to  "never"  with  the  most  frequent  response  being 
"sometimes."  Based  upon  these  responses,  most  decisions  made  by 
personnel  warrants  can  be  said  to  be  routine,  take  1  minute  to  3 
hours  to  make,  and  are  not  creative  in  nature. 

The  applicability  of  functions  performed  was  measured  by 
determining  the  use  of  knowledge,  whether  or  not  functions  were 
decision-oriented  ,  and  the  generic  types  of  functions.  Ninety-one 
percent  of  the  respondents  indicated  that  their  knowledge  of  an  area 
was  used  in  the  performance  of  their  work,  with  the  mean  response 
being  3.4.  The  mean  response  of  work  being  decision-oriented  was 
4.0,  with  96  percent  indicating  a  "sometimes"  to  "always"  response. 
Personnel  warrants  responded  with  a  2.5  or  better  mean  responses 
in  the  following  functions:  diagnosis  (2.66),  debugging  (2.88), 
instruction  (2.67),  analysis  (2.58),  scheduling  (2.55),  and  fault 
isolation  (2.70).  Prediction  and  target  resources  allocation  were  not 
significant  functions  performed  by  personnel  warrants. 
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Comparing  the  profile  to  the  model,  only  one  significant 
difference  is  apparent  -  the  use  of  known  facts.  Facts  were  used  in 
the  thinking  process  but  the  degree  to  which  they  were  used  was 
undetermined.  The  comparison  of  the  measured  variables  to  the 
Expert  System  Application  Criteria  (ESAC)  model  is  in  Appendix  L. 

The  Intelligence  Warrant  Officer  Profile:  The  profile 
developed  on  the  intelligence  warrant  officer  differed  in  many 
respects  from  the  personnel  warrant  officer  profile.  A  sample  size  of 
33  was  analyzed.  The  total  number  of  responses  was  37,  however, 
one  was  deemed  invalid  due  to  incompleteness  and  the  remainder 
due  to  possible  validity  errors.  Analysis  of  valid  survey  responses 
indicated  that  the  thinking  process,  the  decision  characteristics  and 
the  functions  performed  differed  from  the  ESAC  model  in  several  of 
the  areas  measured. 

In  determining  the  use  of  experience  versus  common  sense  in 
the  thinking  process,  61  percent  responded  that  past  experience  was 
the  primary  factor  used  in  decision  making.  However,  like  the 
personnel  respondents,  70  percent  indicated  that  the  extent  of 
experience  used  was  only  "sometimes."  This  response  could  again  be 
the  result  of  poor  survey  questioning.  A  mean  response  of  3.0  in  this 
case  was  inconclusive.  Only  39  percent  of  the  respondents  indicated 
that  common  sense  was  the  primary  factor  used  in  decision  making, 
with  88  percent  responding  that  common  sense  was  used  with  a 
frequency  from  "never"  to  "sometimes."  The  majority  of  the 
respondents  indicated  that  mathematical  models  were  "almost  never" 
used  in  their  decision  making  process.  The  mean  response  to  this 


question  was  1.94.  Only  12  percent  indicated  the  use  of  known  facts 
in  the  decision  making  process,  while  79  percent  indicated  both  facts 
and  intuition  were  used.  This  is  a  repeat  of  the  dilemma  presented  in 
the  personnel  warrants'  result  and  is  explained  in  greater  depth  in 
Chapter  4.  Thus,  it  can  be  concluded  that  experience  is  used  in  the 
decision  making  thinking  process,  common  sense  and  mathematical 
models  are  generally  not  used,  and  known  facts  are  utilized  along 
with  intuition. 

In  determining  the  characteristics  of  decisions  made  by 

intelligence  warrants,  30  percent  indicated  that  decision  making  took 
less  than  1  minute  to  make;  64  percent,  1-59  minutes;  and  6  percent, 
1  -  3  hours  to  make.  None  indicated  that  the  process  took  greater 
than  three  hours.  The  mean  response  to  the  regularity  or  routine 

nature  of  the  decisions  was  3.61.  In  this  case,  the  mode  was  a  3.0,  or 
a  "sometimes"  response,  while  the  median  was  a  4.0,  or  an  "almost 
always"  response.  In  this  case,  however,  91  percent  indicated  that 
this  work  was  creative  in  nature,  ranging  from  "sometimes"  to  "almost 
always".  The  mean  response  was  3.30.  Based  upon  these  responses, 
it  can  be  surmised  that  most  decisions  made  by  intelligence  warrants 
are  routine,  take  1  minute  to  59  minutes  to  make,  and  are  creative  in 
nature. 

In  determining  the  types  of  functions  performed,  91  percent  of 

the  respondents  stated  that  they  used  the  knowledge  they  have  in 

the  performance  of  their  duties.  The  mean  response  was  3.30, 
indicating  more  often  than  "sometimes."  The  mean  response  of  work 
being  decision-oriented  was  3.61,  with  55  percent  responding  that 
their  work  was  "almost  always"  decision-oriented.  Intelligence 
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warrants  responded  with  a  2.5  or  better  response  in  only  one  out  of 
the  eight  functions  measured.  Sixty-one  percent  of  the  respondents 
answered  that  instruction  was  "always"  one  of  the  functions  they 
performed.  A  summation  of  the  intelligence  warrant  officer's  profile 
is  in  Appendix  J. 

Comparing  the  profile  to  the  ESAC  model  (Appendix  L),  it  is 
apparent  that  the  two  do  not  closely  correspond.  The  use  of  known 
facts  is  not  determined  and  creativity  is  a  characteristic  of  the 
decisions  in  the  intelligence  warrant  officer's  decision  making  process. 

The  Supply  Warrant  Officer  Profile:  The  profile  of  the 
supply  warrant  officer  was  developed  using  a  sample  size  of  50.  The 
total  number  of  responses  was  52,  however,  2  were  eliminated  due  to 
incompleteness.  Analysis  of  the  variables  indicated  that  the 

developed  profile  closely  parallels  the  ESAC  model. 

In  determining  the  decision  making  thinking  process,  90 
percent  of  the  respondents  answered  that  past  experience  was 
primarily  used  in  decision  making.  Researching  regulations  and 
manuals  received  4  percent,  and  the  use  of  common  sense  rounded 
out  the  responses  with  6  percent.  The  corresponding  questions  on  the 
extent  of  experience  and  common  sense  use  reinforced  these 
responses.  Ninety-four  percent  indicated  that  experience  was  used 
with  responses  ranging  from  "sometimes"  to  "always."  The  mean  of 
these  responses  was  3.18  indicating  more  often  than  "sometimes." 
Again,  94  percent  indicated  that  common  sense  was  used  from  a 
frequency  of  "never"  to  "sometimes."  Here  the  mean  response  was 
2.80,  indicating  less  often  than  "sometimes."  The  mean  response  to 
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the  use  of  mathematical  models  was  2.72  indicating  less  often  than 
"sometimes."  Eighty-four  percent  indicated  that  models  were  used 
with  the  frequency  of  "never"  to  "sometimes."  The  determination  of 
the  use  of  facts,  for  the  third  time,  indicated  that  68  percent  used 
both  facts  and  intuition  in  the  decision  making  process.  Supply 
warrant  officers  did  have  the  highest  response  (32  percent)  of  the 
three  groups  studied  for  the  use  of  facts.  Based  on  these  results,  the 
decision  making  thinking  process  does  use  experience,  does  not  use 
common  sense  or  mathematical  models,  and  does  use  both  intuition 
and  known  facts. 

In  determining  the  characteristics  of  decisions  made  by  supply 
warrants,  70  percent  of  the  respondents  indicated  that  decisions  took 
from  1  minute  to  59  minutes  to  make.  The  remainder  of  the 
responses  stated  that  decision  making  took  less  than  1  minute.  One 

hundred  percent  of  the  respondents  indicated  that  their  work 

involves  routine  decision  making  with  a  frequency  of  "sometimes"  to 
"always."  The  mean  response  was  4.08  indicating  a  more  often  than 
"sometimes"  response.  Likewise,  creativity  of  decisions  appeared 
insignificant  with  82  percent  of  responses  in  the  range  of  "never"  to 
"sometimes."  The  mean  response,  2.90,  indicated  that  supply  warrant 
officer  work  was  creative  in  nature  less  often  than  "sometimes." 
Based  upon  these  statistics,  it  can  be  concluded  that  decisions  made 
by  supply  warrants  take  1-59  minutes  to  make,  are  routine,  and  are 
not  creative  in  nature. 

In  analyzing  the  functions  performed  by  supply  warrant 

officers,  the  use  of  knowledge  in  their  work  area  had  a  mean 
response  of  3.46,  with  94  percent  ranging  their  responses  from 
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"sometimes"  to  "always,"  Within  the  same  range,  100  percent  of  the 
respondents  indicated  their  work  was  decision-oriented.  The  mean 
response  was  3.82,  the  median  and  mode  both  equalled  4.0  meaning 
an  "almost  always"  response.  Supply  warrant  officers  responded  with 
a  2.5  or  better  mean  in  the  following  functions:  debugging  (2.58), 
instruction  (2.88),  analysis  (2.62),  scheduling  (2.62),  and  fault 
isolation  (2.54).  The  data  measurements  for  each  variable  of  the 
supply  warrant  officer  profile  is  in  Appendix  K. 

Comparing  the  supply  warrant  officer  profile  with  the  model 
(Appendix  L)  shows  the  closeness  of  the  variables.  Only  one  element 
is  different:  use  of  known  facts. 
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CHAPTER  FOUR 

CONCLUSIONS  AND  RECOMMENDATIONS 


POSSIBLE  PROBLEM  AREAS; 

Prior  to  presenting  the  conclusions  and  recommendations  of  this 
paper,  it  is  necessary  to  present  and  discuss  some  of  the  areas  I 
identified  as  possible  problems.  These  problems  have  been  identified 
throughout  the  research  project's  life  cycle  and,  usually,  at  a  point  too 
late  to  rectify  without  added  delay,  cost,  and  possible  incompletion  of 
the  project.  As  a  result,  these  errors  are  inherent  in  the  project,  may 

have  an  affect  on  the  resultant  data,  and  need  to  be  presented.  The 

majority  of  these  errors  are  a  result  of  inexperience.  The  errors, 
some  of  which  have  been  alluded  to  in  the  previous  chapter,  are 
within  the  survey  design.  Please  refer  to  Appendix  B  for  the  survey 
design. 

Problem  Area  One: 

The  question  asked  in  Part  I,  Question  1  of  the  survey 

attempted  to  determine  the  one  element  that  had  the  greatest  impact 

in  the  decision  making  thinking  process.  Because  of  this,  the  question 
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should  have  been  designed  using  ordinal  measurement  in  the  form  of 
rank  order.  Although  I  believe  most  respondents  answered  the 
question  as  intended,  several  added  an  additional  response,  such  as 
"All  of  the  Above",  making  the  surveys  invalid. 

Problem  Area  Two: 

Likewise,  question  3,  Part  I,  should  have  been  rank  ordered 
using  some  type  of  ordinal  measurement.  In  this  question,  the 
opposite  of  Problem  One  occurred.  Because  a  selection  of  "A 
Combination  of  Both"  was  included  and  the  majority  of  the 
respondents  correctly  selected  this  response,  I  was  unable  to  measure 
which  element  -  Known  Facts  or  Intuition  -  had  the  greater  impact  on 
the  decision  making  thinking  process.  Without  this  measurement,  I 
was  unable  to  measure  the  variable  against  the  model. 

Problem  Area  Three: 

Because  of  the  wording  in  statements  2,  5,  and  10  of  Part  II,  the 
validity  checks  built  into  the  system  were  somewhat  insufficient.  For 
example,  statement  2  is:  "I  make  decisions  based  solely  on  past 
experiences."  The  word  "solely"  indicates  that  no  factor  other  than 
experience  is  involved  in  decision  making.  This  was  not  the  intention 
of  the  statement.  The  statement  was  to  have  been  used  as  a  validity 
check  with  Question  1  of  Part  One.  If  a  response  in  question  one  had 
been  "experience,"  then  the  response  to  statement  2  should  have 
ranged  from  "sometimes"  to  "always."  The  term  "solely"  disabled  this 
validity  check  from  effectively  happening.  Again,  I  believe  many 
respondents  answered  the  question  as  I  had  intended.  Several, 
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however,  answered  totally  opposite  as  would  have  been  supposed 
and  a  few  even  made  written  comments.  Because  of  this,  several 
surveys  had  to  be  disregarded  in  the  analysis. 

Allowances  for  these  errors  have  been  made  as  the  situation 
permitted.  Surveys  returned  with  additional  responses  or  without 
questions  answered  were  not  included  in  the  statistical  summary 
process.  Likewise,  questions  that  did  not  meet  the  validity  checks,  as 
inadequate  as  they  were,  were  eliminated.  Other  than  these  two 
precautions,  the  following  conclusions  and  recommendations  were 
based  upon  the  data  provided  by  the  survey.  Any  decisions  made 
based  upon  this  research  project  should  be  made  with  the  realization 
of  possible  errors  and  faulty  analysis. 

CONCLUSIONS 

In  the  area  of  personnel  warrant  officer  expertise,  the  variables 
measured  had  a  90  percent  correspondence  with  the  ESAC  model. 
That  is,  out  of  the  ten  elements  of  the  model,  the  profile  developed  by 
the  analysis  matches  the  model  in  all  areas  with  the  exception  of  one: 
the  use  of  known  facts.  Known  facts  are  used  but  the  extent  of  use  as 
opposed  to  intuition  is  undetermined.  Additionally,  within  the 
element  describing  the  types  of  functions,  the  profile  depicts  six  areas 
in  which  the  type  of  work  performed  by  personnel  warrant  officers 
are  already  successfully  performed  by  expert  system  applications. 
Because  of  this  high  correspondence,  I  believe  the  expertise  of 
personnel  warrant  officers  is  a  prime  candidate  for  an  expert  system 
application.  However,  because  of  the  one  element  deviation,  I  would 
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recommend  further  research  to  clarify  the  use  of  known  facts  before 
resources  are  devoted  to  a  personnel  warrant  officer  application. 

In  the  area  of  intelligence  warrant  officers,  a  greater  deviation 
existed  than  for  that  of  warrants  in  personnel.  The  variables 
measured  only  had  an  80  percent  correspondence  with  the  ESAC 
model.  Out  of  the  ten  elements  of  the  model,  there  was  a  deviation  in 
the  use  of  known  facts  in  the  decision  making  thinking  process  and  in 
the  creativity  of  the  decisions  made.  Additionally,  the  profile  only 
depicts  one  functional  area  (Instruction)  out  of  eight  where 
intelligence  experts  are  currently  working.  Although  this  model 
indicates  that  only  one  of  the  functions  presented  is  necessary  for 
appropriate  expert  system  use,  1  believe  it  is  essential  to  know  what 
percentage  of  an  intelligence  warrant  officer’s  time  is  spent 
performing  this  function.  Taking  all  factors  into  consideration,  1 
believe  the  work  an  intelligence  warrant  officer  does  cannot  be 
augmented  by  an  expert  system  and,  therefore,  recommend  some 
other  viable  augmentation  be  selected  and  researched. 

The  same  conclusion  can  be  drawn  for  supply  warrant  officers 
as  was  drawn  for  the  personnel  warrants.  A  90  percent 
correspondence  between  the  profile  and  the  model  exists,  with  the 
only  variation  being  the  use  of  known  facts  in  the  decision  making 
process.  In  the  area  of  the  functions  performed,  the  profile  presents 
five  areas  in  which  supply  warrant  officer  work  is  similar  to  work 
currently  performed  by  successful  expert  system  applications.  The 
work  performed  by  supply  warrants  is  a  prime  candidate  for  an 
expert  system  application,  once  further  research  has  been  conducted 
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which  proves  the  significant  use  of  known  facts  in  the  decision 
making  process. 

A  final  conclusion  that  can  be  made  is  that  not  all  of  the  Army's 
expertise  is  alike  and,  therefore,  no  blanket  conclusions  can  be  made 
concerning  the  viability  of  expert  systems  to  augment  warrant  officer 
expertise.  Each  area  of  expertise  must  be  analyzed  separate  from  the 
rest  in  order  to  make  valid  conclusions. 


ADDITIONAL  RECOMMENDATIONS 

Besides  the  recommendation  for  additional  research  indicated  in 
the  conclusions,  it  is  important  to  note  that  further  research  would  be 
necessary  before  implementation.  If  a  positive  determination  has 
been  made  concerning  the  use  of  expert  systems  to  augment 
personnel  and  supply  warrant  officer  expertise,  further  research  is 
needed  into  the  most  effective,  productive  and  nondisruptive 
methods  of  implementation.  Like  most  organizations,  the  United 
States  Army  has  limited  resources  and  must  manage  these  effectively 
and  efficiently.  In  addition,  care  must  be  taken  in  the 

implementation  of  computer  systems  which  may  affect  an 
individual's  employment. 
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EXPERT  SYSTEM 

APPLICATION  CRITERIA  MODEL 
(ESAC  Model) 

Application  Criteria  =  Thinking  Process 

+  Decision  Characteristics 
+  Viable  Functions 

Thinking  Process  =  [  Past  Experience  I 

Knowledge  from  Printed 
Material  ] 

+  Known  Facts 

+  No  Use  of  Common  Sense 

+  No  Use  of  Mathematical  Models 

Decision  Characteristics  =  Requires  1  min.  -  3  hrs  to  solve 

+  Routine  Decision-Making 
+  Not  Creative 


Viable  Functions  =  Use  of  Knowledge 

+  Decision-Oriented 
+  [  Fault  Isolation  I  Debugging  I 

Scheduling  I  Prediction  I 
Targeting  Resource  Allocation  I 
Diagnosis  I  Instruction  I 
Analysis  ] 


APPENDIX  A 


QUESTIONNAIRE:  Warrant  Officers'  Decision-Making 


Note:  If  you  are  currently  in  a  branch  immaterial  position,  please  respond  to  questions 
using  your  last  specialty  assignment. 

PART  I:  Please  check  the  one,  most  appropriate  answer  concerning  your  daily  work 
decisions: 

1 .  Most  of  the  decisions  I  make  are  a  result  of  my: 

a.  _  Common  sense. 

b.  _  Experience  in  my  field. 

c.  _  Researching  ARs  and  other  manuals. 

d.  _  None  of  the  above 

2.  Most  of  my  decisions  take: 

a.  _  Less  than  1  minute  to  make. 

b.  _  1  -  59  minutes  to  make. 

c.  _  1  -  3  hours  to  make. 

d.  _  More  than  3  hours  to  make. 

3.  I  make  most  of  my  decisions  using: 

a.  _  Known  facts. 

b.  _  Intuition  (i.e.  personal  insight). 

c.  _  Combination  of  both. 

d.  _  None  of  the  above. 


PART  II:  Please  respond  to  the  following  statements  by  circling  the  appropriate 
number  where: 

1  =  Never,  2  =  Almost  Never,  3  =  Sometimes,  4  =  Almost  Always,  and  5  =  Always. 


1.  My  work  involves  routine  decision-making. 

2.  I  make  decisions  based  solely  on  past 
experiences. 

3.  My  work  is  decision-oriented. 

4.  There  is  not  enough  time  in  the  day  for  me 
to  complete  my  work. 


Never  Almost  Some-  Almost  Always 
Never  limes  Always 

1  2  3  4  5 

1  2  3  4  5 

12  3  4  5 

1  2  3  4  5 


5.  I  make  decisions  based  solely  on  common  sense. 


2  3  4  5 


6.  My  work  is  creative  in  nature. 

7.  In  my  opinion,  there  is  a  difference  in 
knowledge/experience  between 
warrant  officers  of  equal  rank. 

8.  There  is  usually  overlap  time  between 
incoming  and  outgoing  warrant  officers. 


1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 


Please  continue  on  backside. 
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11.  Continued: 


Never  Almost  Some-  Almost  Always 
Never  times  Always 


9.  I  use  mathematical  models  in  my  decision 

making.  1  2  3  4  5 

1 0.  1  make  decisions  based  solely  on  knowledge  1 

have  concerning  my  area  of  expertise.  1  2  3  4  5 


PART  III:  Please  respond  to  the  following  comments  regarding  the  types  of  functions 
you  perform  as  a  warrant  officer  where: 

1  =  Never 

2  =  Sometimes 

3  =  Always 

Never  Some-  Always 
times 


1.  1  predict  likely  consequences  of  given  situations. 

2.  I  make  diagnoses  of  malfunctions  by  observing  people/ 
systems/situations. 

3.  1  develop  or  prescribe  solutions  and  remedies  to  problems. 

4.  1  instruct  or  train  individuals  in  my  area  of  expertise. 

5.  I  conduct  analysis  of  data  and  information. 

6.  I  am  involved  in  the  scheduling  of  resources,  (i.e.  people, 
events,  equipment) 

7.  I  attempt  to  discover  faults  in  systems. 

8.  I  am  responsible  for  targeting  resource  allocation. 


I  2  3 

1  2  3 
1  2  3 
1  2  3 
1  2  3 

1  2  3 
1  2  3 
1  2  3 


PART  IV: 

1. 

2. 

3. 

4. 

5. 


Please  fill  in  the  blanks: 

Age:  _ 

Branch/Specialiy:  _ 

Years  in  Service:  _ 

Number  of  Years  working  in  current  field: 
Number  of  Years  as  a  warrant  officer:  _ 


Thank  you  for  taking  the  time  to  complete  this  questionnaire.  Please  return  it  as  soon  as 
possible  using  the  pre-stamped,  pre-addressed  envelope. 
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SURVEY  DESIGN  —  Element  Measurement 


Number 

1 

Past  Experience  Use 

Part  I,  #1 

Part  II,  #2 

2 

Knowledge  Available 
through  Printed  Materials 

Part  I,  #1 

3 

No  Use  of  Common  Sense 

Parti,  #1 

Pan  II,  #5 

4 

No  Mathematical  Models 

Part  II,  #9 

5 

Decision  Time  Requirements 

Parti,  #2 

6 

Routine  Decision-Making 

Part  II,  #1 

7 

Creativity  of  Decisions 

Part  11,  #6 

8 

Decision-Oriented  Work 

Part  II,  #3 

9 

Use  of  Knowledge 

Part  II,  #10 

10 

Functions  Performed 

Part  III 
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SURVEY  DESIGN  -  Data  Validity  Checks 


Convergent  Validity  Tests 
Test 

Number  A  Reponse  Of:  Indicates 

1  A,  Part  I,  #1 

2  B,  Part  I,  #1 

3  C,  Part  I,  #1 


Discriminant  Validity  Tests 

Test 

Number  A  Reponse  Of:  Indicates 

1  3-5,  Part  II,  #1 

2  1-3,  Part  II,  #6 

3  3-5,  Part  II,  #2 

4  1-3,  Part  II,  #5 


A  Response  Of: 
3-5,  Part  II,  #5 
3-5,  Part  II,  #2 
3-5,  Part  II,  #10 


A  Response  Of: 
1-3,  Part  II,  #6 
3-5,  Part  II,  #1 
1-3,  Part  II,  #5 
3-5,  Part  II,  #2 
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SAMPLING  PROCESS  -  Statistics 


rs 

PS 

a 

OO 

c 

”cL 

E 

cS 

on 


PS 

E 

<u 

C/3 

>> 

C/5 


c  © 

c  a. 

•o 
c 

PS  PS 

“  on 


00 

c 

^  .© 

i « 


C/3; 

<u 

C/3| 

c 

o 

Q 

C/3 

O 


PS 

> 


c/l 

<D 

C/2 


O 

a. 

£  <S3 

a: 


o  -2 

w 

c  -g 

O  Q, 

w-  o 

£  <£ 


« 

PS 

Q 

00 
c 

"a. 

E 

to 

on 

c 
o 

*w  CO 


JO 
CL 

i  «i 

c 
o 


.2 

a 

CL 

o 

a. 


a.. 

c2  $ 


o 

•n 

JPS 

3 

D 

O 

CL 


<D 

E 

4> 

E 

eo 

z 

E 

cd 

PS 

Z 

■a 

Z 

c 

*-• 

o 

c/5 

C/5 

o 

w- 

V 

IE 

E 

on 

*71 

ri 

67 

33 

50 

150 

$ 

NO 

o 

r-~ 

m 

in 

vn 

in 

oo 

CT' 

r- 

cT 

VO 

r- 

m 

© 

© 

O 

m 

co 

m 

m 

m 

m 

OO 

m 

CN 

ON 

— 

P~ 

On 

(N 

00 

■** 

<N 

■*t 

O 

ON 

1— 

m 

00 

u 

u 

a> 

c 

c 

e 

V 

00 

tpl 

_>V 

o 

C/3 

V« 

u 

"v 

«-» 

c 

"a 

a 

3 

c 

a. 

on 

H 

APPENDIX  E 


ANALYSIS  --  Personnel  Warrant  Officer  Sample  Data  Tabulation  and  Summary  Statistics 
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Item  Variable _ Scale  Responses _ Freq.  Cum/Freq%  %  Stats 
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ANALYSIS  --  Intelligence  Warrant  Officer  Sample  Data  Tabulation  and  Summary  Statistics 
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Item  Variable _ Scale  Responses _ Freq.  Cum/Freq% _ %  Stats 


r}- 


a  -a 

<u  <u 

2  2 


co 


CS 


<u 

■g 


II  VO 
l/~> 

a)  ’ 
oo  II 
c  _ 
a  Q 
06  on 


(S 

Tt;  cn 

<s 


CO 


CO 


VO 
4>  ‘ 

oo  || 

s  s  s  I  s 


c  " 

C  .5  4) 

a  -o  -o 

4>  4>  O 


r- 

<s  CN 

r4 


co 


II 


II  ^ 
-  II 


C  .2  4> 


r~~ 


a  *d  tj  11 
o  u  o  =  Q 

s  s  s  S  8 


Ov  <S 


CO 


II 


II 


.5  4) 


c  . 
a  "O  "O 
4i  4>  O 


II 

vO 
4)  ' 

oo  || 

a  Q 

06  oo 


Tt  o 

vO  vO  CO 


m  Ov 
vO  rf  rt 


—  CO 
O  vO  co 


—  —  00 
<S  vO  — • 


\0  r-- 


O 

—  o 

O  1C,  - 


o 
r-  o 

sO  v©  — 


o 

-no 

OO  — 


Ol  O)  — 


m  vo 


O  — 

n  n  - 


oj  vo 


—  oi  CO 


—  (N  CO 


—  n  co 


—  n  co 


C/5 

OO 

c 

c 

4> 

2 

3 

o 

CO 

«  C 

06  O 

C fi 

"a 

o 

o  u 

■o 

4m* 

oo  5 

mam 

4) 

•M  *3 

t- 

C8 

C 

.C 

U 

a  © 

£  £ 

a  — 

E-  < 

< 

oo 

vO 

oo 

— « 

— 

! 


APPENDIX  G 


(N 

00 

1 

o 

o 

Tj* 

co 

VO 

OO 

Tt- 

OO 

co 

l 

On 

CO 

1 

CO 

1 

rt 

CO 

CO 

II 

no 

<s 

II 

CO 

r4 

II 

CO 

04 

II 

CO 

CO 

II 

CO 

II 

c 

II 

II 

69 

II 

c 

II 

ll 

S 

II 

c 

II 

II 

r- 

II 

c 

II 

II 

00 

II 

C 

II 

</) 

CG 

c 

S3 

V 

S3 

■O 

<u 

<u 

T3 

O 

<D 

co 

c 

CC 

ii 

o 

c 

S3 

0) 

S3 

’-3 

<D 

O 

T3 

O 

<D 

00 

c 

II 

o 

c 

S3 

O 

S3 

'S 

O 

"8 

V 

00 

c 

S3 

II 

r\ 

c 

S3 

<D 

_S3 

V 

<D 

03 

O 

u 

60 

c 

S3 

II 

Q 

c 

S3 

<D 

2 

'■5 

0> 

u 

■g 

£5 

£ 

£ 

ck: 

UJ 

00 

s 

QC 

00 

S 

Q£ 

U-J 

00 

£ 

O' 

00 

£ 

_  CN 

Tt 

o 

vO 

00 

(N 

■3f 

OO 

00 

<N 

vO 

vO 

VO 

o 

o 

CO 

I/O 

— 

Tj- 

— 

r- 

vO 

o 

<s 

oo 

— 

O 

VO 

CO 

■'t 

o 

<N 

Tf- 

&ci 

O' 

<u 

l— 

Ll. 

E 

3 

u 


3 

U- 


<D 

v. 

C 

o 

c. 

■s. 

<D 

a: 


— 

S3 

o 

00 


_4> 

-O 

2 

u. 

s3 

> 


C 

o 

©  O  eo  oc  — 


r-  i/o 

C  O  -  M  X 


O  O 
(N  ^  C  C 
rsi  On  —  — 


vO 

NafnnC 


c  o 

vC  N  c  C 

N  N  M  —  - 


<N  0C 

, - NO>C 


o  o 

oc  rt  O  O 
vO  co  OC  —  — 


vO  co 

ro  —  m  oo  © 


-  M  (O  it, 


-  (S  n  ^  it,  —  n  n  t  ir,  -  n  m  vt  in 


IT) 


II  VO 
i) 

w>  ll 

c  A 

S3  Q 
CC  00 


Tt  VO 


o 

N  't  O 
<N  vO  m  ON  — 


co  *-• 

-  N  N  M  m 


-  N  m  Tt  in 


u 

c 

W 

u 

S/5 

a 

CG 

'C 

<u 

U 

9 

00 

>v 

u 

60 

c 

c 

w 

CG 

E  c* 

1 o 

o 

o 

> 

a> 

!/5 

E 

«  "G 

<*-  •“ 
o  £ 

G 

4) 

E  « 

O  t/5 

<9 

a> 

u 

t3  T3 

S3  O 

W  o 

3*  c 

Q 

U  D 

u 

S  5 

D  * 

o 

VO 

p- 

00 

ON 

wm 

APPENDIX  H 


C/5 

«3 


$ 


© 

3' 

U 

i— 

LL 

P 

3 

u 


B 

X: 


V 

V 

c 

o 

Q, 

■v. 

0>| 

DC 


JJ 

cs 

u 

on 


_4> 

-O 

2 

u. 

« 

> 

E 

4> 


o 

<N 

m 

i 

<N 

m 

i 

00 

m 

i 

00 

r*n 

1 

wo 

m 

oo 

m 

ci 

II 

<N 

<N 

(N 

II 

(S 

ri 

II 

r«n 

<s 

fNJ 

II 

m 

C4 

II 

C 

II 

II 

49 

II 

C 

II 

n 

54 

II 

c 

II 

II 

50 

II 

c 

II 

II 

33 

c 

3 

« 

‘■5 

<D 

-o 

<U 

00 

n 

C 

« 

3 

■5 

u 

V 

O 

oo 

n 

c 

ea 

2 

’■5 

U 

•o 

<D 

00 

gg 

n 

c 

PS 

pi 

'S 

4> 

TD 

<u 

oo 

n 

<U 

<L> 

o 

PS 

04 

Q 

00 

1) 

£ 

u 

£ 

o 

2 

P3 

04 

a 

on 

V 

<u 

O 

£ 

« 

04 

O 

on 

<D 

u 

£ 

o 

s 

ts 

04 

a 

on 

c 

c 

(N 

NO 

(N 

00 

r-i 

oo 

O 

NO 

tj- 

<N 

wn 

© 

wo 

o 

— 

oc 

C 

c  o 

NO  — 


©  r*1  M 


—  r  )  r<~, 


Tf 

ir-, 


nO 

—  M  (N 


r)  c 

C  Tt  - 


—  ON 
O  <N 


—  N  m 


—  N  r*; 


o 

o 


rf 

O  NO  rt 


—  r-i  m 


oo 

c 

c 

o 

o 

Vi 

c 

*—> 

*S5 

‘ob 

o 

O 

■5 

w 

c 

oo 

00 

3 

ft 

3 

fa. 

<o 

•u 

u 

a 

I/! 

fa. 

0. 

5 

c 

mm 

n 

m 

■"'fr 

mm 

mm 

APPENDIX  H 
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ANALYSIS 


Personnel  Warrant  Officers’  Sample  Profile 
Sample  Size  =  67 


Item 

Number _ Profile  Element _ Data  Measurement 


1 

Thinking  Process 

Experience 

2 

Time  Factor 

1-59  mins. 

3 

Use  of  Facts 

Facts  &  Intuition 

4 

Regularity 

4 

5 

Use  of  Experience 

3 

6 

Decision-Oriented 

4 

7 

Use  of  Common  Sense 

2.75 

8 

Creativity 

2.97 

9 

Mathematical  Models 

2.58 

1  0 

Use  of  Knowledge 

3.40 

1  1 

Prediction 

2.40 

1  2 

Diagnosis 

2.66 

1  3 

Debugging 

2.88 

14 

Instruction 

2.67 

1  5 

Analysis 

2.58 

1  6 

Scheduling 

2.55 

1  7 

Fault  Isolation 

2.70 

1  8 

Target  Resource  Allocation 

2.31 

APPENDIX  I 


ANALYSIS  - 

Intelligence  Warrant  Officers’ 

Sample  Profile 

Sample  Size  =  33 

Item 

Number 

Profile  Element 

1 

Thinking  Process 

Experience 

2 

Time  Factor 

1-59  mins. 

3 

Use  of  Facts 

Facts  &  Intuition 

4 

Regularity 

3.61 

5 

Use  of  Experience 

3 

6 

Decision -Oriented 

3.61 

7 

Use  of  Common  Sense 

2.97 

8 

Creativity 

3.30 

9 

Mathematical  Models 

1.94 

1  0 

Use  of  Knowledge 

3.30 

1  1 

Prediction 

2.27 

1  2 

Diagnosis 

2.30 

1  3 

Debugging 

2.48 

1  4 

Instruction 

2.56 

1  5 

Analysis 

2.24 

1  6 

Scheduling 

2.42 

17 

Fault  Isolation 

2.27 

18 

Target  Resource  Allocation 

1.97 

APPENDIX  J 


ANALYSIS  - 

Item 

Number 

Supply  Warrant  Officers' 

Sample  Size  =  50 

Profile  Element 

Sample  Profile 

Data  Measurement 

1 

Thinking  Process 

Experience 

2 

Time  Factor 

1-59  mins. 

3 

Use  of  Facts 

Facts  &  Intuition 

4 

Regularity 

4.08 

5 

Use  of  Experience 

3.18 

6 

Decision -Oriented 

3.82 

7 

Use  of  Common  Sense 

2.80 

8 

Creativity 

2.90 

9 

Mathematical  Models 

2.72 

1  0 

Use  of  Knowledge 

3.46 

1  1 

Prediction 

2.40 

1  2 

Diagnosis 

2.44 

1  3 

Debugging 

2.58 

14 

Instruction 

2.88 

15 

Analysis 

2.62 

1  6 

Scheduling 

2.62 

1  7 

Fault  Isolation 

2.54 

1  8 

Target  Resource  Allocation  2.38 

APPENDIX  K 


ANALYSIS  --  Comparison  of  Sample  Populations 


Item  Model 
#  Variable 

Model 

Measure 

mm 

1  Thinking 

Experience/ 

Y 

Y 

Y 

Process 

Research 

Use  of  Facts 

N 

N 

N 

No  Common  Sense 

Y 

Y 

Y 

No  Mathematical 
Models 

Y 

Y 

Y 

2  Decision 

lmin-3  hrs 

Y 

Y 

Y 

Characteristics 

Routine  Decisions 

Y 

Y 

Y 

Not  Creative 

Y 

N 

Y 

3  Functions 

Use  of  Knowledge 

Y 

Y 

Y 

Decision-Oriented 

Y 

Y 

Y 

Prediction/ 

N 

N 

N 

Diagnosis/ 

Y 

N 

N 

Debugging/ 

Y 

N 

Y 

Instruction/ 

Y 

Y 

Y 

Analysis/ 

Y 

N 

Y 

Scheduling/ 

Y 

N 

Y 

Fault  Isolation/ 

Y 

N 

Y 

Target  Resource 

N 

N 

N 
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